[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:24 dist-test-slave-ss4n krb5kdc[1808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:24 dist-test-slave-ss4n krb5kdc[1808](info): set up 2 sockets
Aug 08 10:39:24 dist-test-slave-ss4n krb5kdc[1808](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 10:39:25.185894 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:25 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:25.321350 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:36899
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:36899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:25.438515 1821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:25.438841 1821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:25.438882 1821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:25.442929 1821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:25.442989 1821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:25.443008 1821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:25.443025 1821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:25.443043 1821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:25.448441 1821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:36899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:36899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:25.450693 1821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:25.452101 1821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:25.464570 1829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:25.465644 1827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:25.465703 1826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:25.466729 1821 server_base.cc:1034] running on GCE node
I20260808 10:39:25.467387 1821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:25.469162 1821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:25.470371 1821 hybrid_clock.cc:648] HybridClock initialized: now 1786185565470355 us; error 61 us; skew 500 ppm
Aug 08 10:39:25 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:25.473388 1821 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:25.474635 1821 webserver.cc:460] Webserver started at http://127.1.194.126:42207/ using document root <none> and password file <none>
I20260808 10:39:25.475456 1821 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:25.475553 1821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:25.475934 1821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:25.478521 1821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "8e856a41ac644979a5d3bf00729c3f28"
format_stamp: "Formatted at 2026-08-08 10:39:25 on dist-test-slave-ss4n"
server_key: "74ab3a082ccbdaa0b023f25419a26afd"
server_key_iv: "edd79face6331d31a5c9bc9e9b17571e"
server_key_version: "encryptionkey@0"
I20260808 10:39:25.479080 1821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "8e856a41ac644979a5d3bf00729c3f28"
format_stamp: "Formatted at 2026-08-08 10:39:25 on dist-test-slave-ss4n"
server_key: "74ab3a082ccbdaa0b023f25419a26afd"
server_key_iv: "edd79face6331d31a5c9bc9e9b17571e"
server_key_version: "encryptionkey@0"
I20260808 10:39:25.484180 1821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 10:39:25.487639 1837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:25.489432 1821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:39:25.489730 1821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "8e856a41ac644979a5d3bf00729c3f28"
format_stamp: "Formatted at 2026-08-08 10:39:25 on dist-test-slave-ss4n"
server_key: "74ab3a082ccbdaa0b023f25419a26afd"
server_key_iv: "edd79face6331d31a5c9bc9e9b17571e"
server_key_version: "encryptionkey@0"
I20260808 10:39:25.489871 1821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:25.512177 1821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:25.516003 1821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:25.516294 1821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:25.526338 1821 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:36899
I20260808 10:39:25.526384 1893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:36899 every 8 connection(s)
I20260808 10:39:25.527724 1821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:25.529021 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 1821
I20260808 10:39:25.529202 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:25.530748 1801 external_mini_cluster.cc:1428] Setting key 5e81102206e1f08a9a09d87e338840d7
I20260808 10:39:25.532891 1894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:25.542429 1894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28: Bootstrap starting.
I20260808 10:39:25.546277 1894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:25.547452 1894 log.cc:824] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:25.550334 1894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28: No bootstrap required, opened a new log
Aug 08 10:39:25 dist-test-slave-ss4n krb5kdc[1808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:25.554901 1894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e856a41ac644979a5d3bf00729c3f28" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36899 } }
I20260808 10:39:25.555233 1894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:25.555351 1894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e856a41ac644979a5d3bf00729c3f28, State: Initialized, Role: FOLLOWER
I20260808 10:39:25.556185 1894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e856a41ac644979a5d3bf00729c3f28" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36899 } }
I20260808 10:39:25.556403 1894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:25.556545 1894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:25.556706 1894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:25.557754 1894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e856a41ac644979a5d3bf00729c3f28" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36899 } }
I20260808 10:39:25.558337 1894 leader_election.cc:302] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e856a41ac644979a5d3bf00729c3f28; no voters:
I20260808 10:39:25.558832 1894 leader_election.cc:288] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:25.559185 1899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:25.559474 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:25.535391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.1.194.126:36899)
0808 10:39:25.535885 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:25.535902 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:25.537172 (+ 1270us) server_negotiation.cc:407] Connection header received
0808 10:39:25.540580 (+ 3408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:25.540596 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:25.540991 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:25.541328 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:25.543810 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:25.544962 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:25.546322 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:25.546894 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:25.547081 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:25.551863 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:25.551900 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:25.551913 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:25.551942 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:25.554922 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:25.555728 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:25.555733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:25.555738 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:25.555856 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:25.556259 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:25.556269 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:25.556271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:25.556912 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:25.557141 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:25.557846 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 10:39:25.558157 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":170,"threads_started":1}
I20260808 10:39:25.559562 1899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [term 1 LEADER]: Becoming Leader. State: Replica: 8e856a41ac644979a5d3bf00729c3f28, State: Running, Role: LEADER
I20260808 10:39:25.560277 1899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e856a41ac644979a5d3bf00729c3f28" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36899 } }
I20260808 10:39:25.560663 1894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:25.562212 1901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e856a41ac644979a5d3bf00729c3f28. Latest consensus state: current_term: 1 leader_uuid: "8e856a41ac644979a5d3bf00729c3f28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e856a41ac644979a5d3bf00729c3f28" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36899 } } }
I20260808 10:39:25.562337 1901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:25.562947 1908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:25.566377 1900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e856a41ac644979a5d3bf00729c3f28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e856a41ac644979a5d3bf00729c3f28" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36899 } } }
I20260808 10:39:25.566527 1900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:25.570399 1908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:25.579392 1908 catalog_manager.cc:1347] Generated new cluster ID: 4049e3d786c04c9a95ed3d0492537ea4
I20260808 10:39:25.579474 1908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:25.588730 1908 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:25.589841 1908 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:25.599697 1908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e856a41ac644979a5d3bf00729c3f28: Generated new TSK 0
I20260808 10:39:25.600548 1908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:25.636940 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36899
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:25.785806 1921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:25.786106 1921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:25.786156 1921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:25.790213 1921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:25.790288 1921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:25.790354 1921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:25.795985 1921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:25.797132 1921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:25.798169 1921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:25.809337 1926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:25.809310 1927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:25.809299 1929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:25.809700 1921 server_base.cc:1034] running on GCE node
I20260808 10:39:25.810153 1921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:25.810757 1921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:25.811928 1921 hybrid_clock.cc:648] HybridClock initialized: now 1786185565811905 us; error 33 us; skew 500 ppm
Aug 08 10:39:25 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:25.815127 1921 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:25.816010 1921 webserver.cc:460] Webserver started at http://127.1.194.65:37993/ using document root <none> and password file <none>
I20260808 10:39:25.816552 1921 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:25.816613 1921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:25.816778 1921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:25.818867 1921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "78392b275aba48ac9521abd516ef2713"
format_stamp: "Formatted at 2026-08-08 10:39:25 on dist-test-slave-ss4n"
server_key: "36a9448f4c10c3bb9c00affe15567c75"
server_key_iv: "2a59b51f9e496611e9caeb7d0b58d43c"
server_key_version: "encryptionkey@0"
I20260808 10:39:25.819401 1921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "78392b275aba48ac9521abd516ef2713"
format_stamp: "Formatted at 2026-08-08 10:39:25 on dist-test-slave-ss4n"
server_key: "36a9448f4c10c3bb9c00affe15567c75"
server_key_iv: "2a59b51f9e496611e9caeb7d0b58d43c"
server_key_version: "encryptionkey@0"
I20260808 10:39:25.823496 1921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:25.826400 1936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:25.827701 1921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:25.827840 1921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "78392b275aba48ac9521abd516ef2713"
format_stamp: "Formatted at 2026-08-08 10:39:25 on dist-test-slave-ss4n"
server_key: "36a9448f4c10c3bb9c00affe15567c75"
server_key_iv: "2a59b51f9e496611e9caeb7d0b58d43c"
server_key_version: "encryptionkey@0"
I20260808 10:39:25.827963 1921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:25.842216 1921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:25.845818 1921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:25.846038 1921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:25.846803 1921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:25.847918 1921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:25.847980 1921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:25.848057 1921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:25.848090 1921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:25.860018 1921 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:36655
I20260808 10:39:25.860037 2055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:36655 every 8 connection(s)
I20260808 10:39:25.861089 1921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:25.865582 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 1921
I20260808 10:39:25.865680 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:25.865938 1801 external_mini_cluster.cc:1428] Setting key 1c836ea5663ae991b62a85d43f7c565f
Aug 08 10:39:25 dist-test-slave-ss4n krb5kdc[1808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:25.877921 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:25.863653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:53929 (local address 127.1.194.126:36899)
0808 10:39:25.863826 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:25.863831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:25.864882 (+ 1051us) server_negotiation.cc:407] Connection header received
0808 10:39:25.866532 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:25.866538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:25.866598 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:25.866709 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:25.868184 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:25.869036 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:25.870335 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:25.870774 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:25.870910 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:25.873797 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:25.873804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:25.873810 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:25.873847 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:25.875710 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:25.876260 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:25.876265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:25.876267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:25.876366 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:25.876724 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:25.876729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:25.876731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:25.876981 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:25.877081 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:25.877672 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 10:39:25.877791 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:39:25.878834 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:25.864113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:36899 (local address 127.1.194.65:53929)
0808 10:39:25.864714 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:25.864758 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:25.866272 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:25.866920 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:25.866928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:25.867459 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:25.868016 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:25.868037 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:25.869179 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:25.869182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:25.870207 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:25.870217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:25.870975 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:25.870981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:25.871248 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:25.871955 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:25.871972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:25.873621 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:25.875841 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:25.875845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:25.875858 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:25.876134 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:25.876460 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:25.876462 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:25.876463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:25.876607 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:25.877092 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:25.877097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:25.877380 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 10:39:25.877588 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 10:39:25.877852 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:39:25.880232 2056 heartbeater.cc:347] Connected to a master server at 127.1.194.126:36899
I20260808 10:39:25.880486 2056 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:25.880980 2056 heartbeater.cc:511] Master 127.1.194.126:36899 requested a full tablet report, sending...
I20260808 10:39:25.883193 1854 ts_manager.cc:194] Registered new tserver with Master: 78392b275aba48ac9521abd516ef2713 (127.1.194.65:36655)
I20260808 10:39:25.884665 1854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:53929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:25.897243 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36899
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:26.037222 2061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:26.037562 2061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:26.037667 2061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:26.041831 2061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:26.041924 2061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:26.042021 2061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:26.047551 2061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:26.048756 2061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:26.049865 2061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:26.059641 2066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:26.059751 2061 server_base.cc:1034] running on GCE node
W20260808 10:39:26.059791 2070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:26.059886 2067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:26.060297 2061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:26.060989 2061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:26.062283 2061 hybrid_clock.cc:648] HybridClock initialized: now 1786185566062264 us; error 30 us; skew 500 ppm
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:26.065243 2061 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:26.066186 2061 webserver.cc:460] Webserver started at http://127.1.194.66:35169/ using document root <none> and password file <none>
I20260808 10:39:26.066753 2061 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:26.066821 2061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:26.067114 2061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:26.069303 2061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "474236751abe4c779f1dddf52f076d84"
format_stamp: "Formatted at 2026-08-08 10:39:26 on dist-test-slave-ss4n"
server_key: "5df0c97d81f0fb00e6bac4a0bbb1ff36"
server_key_iv: "19d3cbbf40427776790777dcfd5ca0c1"
server_key_version: "encryptionkey@0"
I20260808 10:39:26.069974 2061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "474236751abe4c779f1dddf52f076d84"
format_stamp: "Formatted at 2026-08-08 10:39:26 on dist-test-slave-ss4n"
server_key: "5df0c97d81f0fb00e6bac4a0bbb1ff36"
server_key_iv: "19d3cbbf40427776790777dcfd5ca0c1"
server_key_version: "encryptionkey@0"
I20260808 10:39:26.073968 2061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:39:26.076377 2077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:26.077602 2061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:26.077731 2061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "474236751abe4c779f1dddf52f076d84"
format_stamp: "Formatted at 2026-08-08 10:39:26 on dist-test-slave-ss4n"
server_key: "5df0c97d81f0fb00e6bac4a0bbb1ff36"
server_key_iv: "19d3cbbf40427776790777dcfd5ca0c1"
server_key_version: "encryptionkey@0"
I20260808 10:39:26.077842 2061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:26.089553 2061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:26.092752 2061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:26.092954 2061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:26.093659 2061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:26.094506 2061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:26.094575 2061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:26.094646 2061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:26.094686 2061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:26.106110 2061 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:37937
I20260808 10:39:26.106141 2197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:37937 every 8 connection(s)
I20260808 10:39:26.107234 2061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:26.109812 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2061
I20260808 10:39:26.109943 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:26.110276 1801 external_mini_cluster.cc:1428] Setting key 77dae357abdad12acc90ee8a919bd51c
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:26.124076 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:26.109149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:52715 (local address 127.1.194.126:36899)
0808 10:39:26.109297 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:26.109301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:26.110330 (+ 1029us) server_negotiation.cc:407] Connection header received
0808 10:39:26.112076 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:26.112079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:26.112124 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:26.112194 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:26.113885 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.114409 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.116200 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.116427 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.116511 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:26.119594 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:26.119599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:26.119604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:26.119634 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:26.121561 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.122207 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.122211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.122213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.122344 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.122885 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.122888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.122889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.123100 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:26.123219 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:26.123760 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 10:39:26.123889 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:39:26.124997 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:26.109621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:36899 (local address 127.1.194.66:52715)
0808 10:39:26.110141 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:26.110203 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:26.111846 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:26.112419 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:26.112428 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:26.112983 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:26.113723 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:26.113747 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.114543 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.114549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:26.116061 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:26.116077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.116591 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.116598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:26.116830 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:26.117568 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:26.117597 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:26.119411 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:26.121706 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:26.121715 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:26.121727 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:26.122023 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:26.122556 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:26.122559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:26.122560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:26.122708 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:26.123217 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:26.123224 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:26.123485 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 10:39:26.123686 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 10:39:26.124022 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:26.126474 2198 heartbeater.cc:347] Connected to a master server at 127.1.194.126:36899
I20260808 10:39:26.126739 2198 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:26.127344 2198 heartbeater.cc:511] Master 127.1.194.126:36899 requested a full tablet report, sending...
I20260808 10:39:26.128618 1854 ts_manager.cc:194] Registered new tserver with Master: 474236751abe4c779f1dddf52f076d84 (127.1.194.66:37937)
I20260808 10:39:26.129273 1854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:52715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:26.144233 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36899
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:26.265940 2203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:26.266188 2203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:26.266265 2203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:26.270430 2203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:26.270493 2203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:26.270586 2203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:26.276124 2203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:26.277189 2203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:26.278276 2203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:26.288569 2209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:26.288635 2212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:26.288546 2210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:26.288911 2203 server_base.cc:1034] running on GCE node
I20260808 10:39:26.289318 2203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:26.290005 2203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:26.291179 2203 hybrid_clock.cc:648] HybridClock initialized: now 1786185566291104 us; error 94 us; skew 500 ppm
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:26.294173 2203 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:26.295055 2203 webserver.cc:460] Webserver started at http://127.1.194.67:33825/ using document root <none> and password file <none>
I20260808 10:39:26.295620 2203 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:26.295706 2203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:26.295940 2203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:26.298079 2203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "a63f39a520164e4883fa716a1407bee4"
format_stamp: "Formatted at 2026-08-08 10:39:26 on dist-test-slave-ss4n"
server_key: "5fd393c0f21ca3691d64c75d0cb6cfa0"
server_key_iv: "afd32e98521cbf78bd1a616dff4e1990"
server_key_version: "encryptionkey@0"
I20260808 10:39:26.298655 2203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "a63f39a520164e4883fa716a1407bee4"
format_stamp: "Formatted at 2026-08-08 10:39:26 on dist-test-slave-ss4n"
server_key: "5fd393c0f21ca3691d64c75d0cb6cfa0"
server_key_iv: "afd32e98521cbf78bd1a616dff4e1990"
server_key_version: "encryptionkey@0"
I20260808 10:39:26.302416 2203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:39:26.304781 2220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:26.306056 2203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:26.306185 2203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "a63f39a520164e4883fa716a1407bee4"
format_stamp: "Formatted at 2026-08-08 10:39:26 on dist-test-slave-ss4n"
server_key: "5fd393c0f21ca3691d64c75d0cb6cfa0"
server_key_iv: "afd32e98521cbf78bd1a616dff4e1990"
server_key_version: "encryptionkey@0"
I20260808 10:39:26.306305 2203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:26.319844 2203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:26.323096 2203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:26.323311 2203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:26.324107 2203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:26.324956 2203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:26.325028 2203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:26.325102 2203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:26.325145 2203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:26.335889 2203 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:35265
I20260808 10:39:26.335919 2340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:35265 every 8 connection(s)
I20260808 10:39:26.336846 2203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:26.341504 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2203
I20260808 10:39:26.341638 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:26.341938 1801 external_mini_cluster.cc:1428] Setting key 75f9b9ead8368943374eed77269ce58a
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:26.351287 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:26.338749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:41787 (local address 127.1.194.126:36899)
0808 10:39:26.338900 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:26.338905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:26.339757 (+ 852us) server_negotiation.cc:407] Connection header received
0808 10:39:26.341080 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:26.341083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:26.341128 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:26.341207 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:26.342898 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.343628 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.345127 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.345342 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.345425 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:26.348042 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:26.348044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:26.348046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:26.348071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:26.349416 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.349962 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.349965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.349966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.350044 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.350367 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.350369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.350370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.350538 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:26.350617 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:26.351064 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 10:39:26.351168 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:39:26.352074 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:26.339174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:36899 (local address 127.1.194.67:41787)
0808 10:39:26.339611 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:26.339652 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:26.340885 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:26.341430 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:26.341439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:26.342061 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:26.342738 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:26.342760 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.343814 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.343816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:26.345015 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:26.345025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.345572 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.345576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:26.345720 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:26.346311 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:26.346327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:26.347892 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:26.349604 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:26.349608 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:26.349620 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:26.349873 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:26.350135 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:26.350137 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:26.350139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:26.350278 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:26.350634 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:26.350639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:26.350859 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 10:39:26.351009 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 10:39:26.351259 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":103,"threads_started":1}
I20260808 10:39:26.353287 2341 heartbeater.cc:347] Connected to a master server at 127.1.194.126:36899
I20260808 10:39:26.353632 2341 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:26.354118 2341 heartbeater.cc:511] Master 127.1.194.126:36899 requested a full tablet report, sending...
I20260808 10:39:26.355293 1854 ts_manager.cc:194] Registered new tserver with Master: a63f39a520164e4883fa716a1407bee4 (127.1.194.67:35265)
I20260808 10:39:26.356071 1854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:41787
I20260808 10:39:26.356500 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:26.386083 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:26.376015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.1.194.65:36655)
0808 10:39:26.376326 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:26.376340 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:26.376474 (+ 134us) server_negotiation.cc:407] Connection header received
0808 10:39:26.376606 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:26.376612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:26.376809 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:26.376949 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:26.377689 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.378506 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.379419 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.379856 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.379947 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:26.382633 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:26.382642 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:26.382656 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:26.382679 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:26.384420 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.384840 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.384845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.384847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.384921 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.385151 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.385153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.385154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.385380 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:26.385599 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:26.385765 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 10:39:26.385891 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
W20260808 10:39:26.387140 1966 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46306
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[1808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:26.416123 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:26.406884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.1.194.65:36655)
0808 10:39:26.407071 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:26.407077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:26.407241 (+ 164us) server_negotiation.cc:407] Connection header received
0808 10:39:26.407417 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:26.407420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:26.407470 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:26.407558 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:26.408264 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.408792 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.409826 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:26.410043 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:26.410160 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:26.412924 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:26.412926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:26.412928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:26.412954 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:26.414337 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.414804 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.414806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.414808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.414886 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:26.415162 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:26.415164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:26.415165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:26.415359 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:26.415523 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:26.415816 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 10:39:26.415919 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 10:39:26.417379 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 1921
I20260808 10:39:26.424315 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2061
I20260808 10:39:26.431480 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2203
I20260808 10:39:26.438407 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 1821
2026-08-08T10:39:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2408 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[2356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[2356](info): set up 2 sockets
Aug 08 10:39:26 dist-test-slave-ss4n krb5kdc[2356](info): commencing operation
krb5kdc: starting...
W20260808 10:39:27.527292 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:27 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:27.656163 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:44449
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:44449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:27.771006 2371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:27.771325 2371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:27.771399 2371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:27.775488 2371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:27.775583 2371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:27.775624 2371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:27.775682 2371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:27.775720 2371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:27.781157 2371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:44449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:44449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:27.782296 2371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:27.783325 2371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:27.791893 2377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:27.791909 2379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:27.791908 2376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:27.792104 2371 server_base.cc:1034] running on GCE node
I20260808 10:39:27.792722 2371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:27.793892 2371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:27.795110 2371 hybrid_clock.cc:648] HybridClock initialized: now 1786185567795082 us; error 48 us; skew 500 ppm
Aug 08 10:39:27 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:27.801321 2371 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:27.802275 2371 webserver.cc:460] Webserver started at http://127.1.194.126:43877/ using document root <none> and password file <none>
I20260808 10:39:27.802884 2371 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:27.803027 2371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:27.803264 2371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:27.805476 2371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "377fef551f5640a0af45ae31ee3d0d2b"
format_stamp: "Formatted at 2026-08-08 10:39:27 on dist-test-slave-ss4n"
server_key: "11c2e3e8accde745302941ac97cddc2f"
server_key_iv: "474d40698a4dd49660e320c7b79201c8"
server_key_version: "encryptionkey@0"
I20260808 10:39:27.806125 2371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "377fef551f5640a0af45ae31ee3d0d2b"
format_stamp: "Formatted at 2026-08-08 10:39:27 on dist-test-slave-ss4n"
server_key: "11c2e3e8accde745302941ac97cddc2f"
server_key_iv: "474d40698a4dd49660e320c7b79201c8"
server_key_version: "encryptionkey@0"
I20260808 10:39:27.809932 2371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:39:27.812285 2386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:27.813612 2371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:27.813750 2371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "377fef551f5640a0af45ae31ee3d0d2b"
format_stamp: "Formatted at 2026-08-08 10:39:27 on dist-test-slave-ss4n"
server_key: "11c2e3e8accde745302941ac97cddc2f"
server_key_iv: "474d40698a4dd49660e320c7b79201c8"
server_key_version: "encryptionkey@0"
I20260808 10:39:27.813863 2371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:27.828377 2371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:27.831665 2371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:27.831866 2371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:27.839403 2371 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:44449
I20260808 10:39:27.839478 2440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:44449 every 8 connection(s)
I20260808 10:39:27.840358 2371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:27.843257 2441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:27.843326 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2371
I20260808 10:39:27.843436 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:27.843726 1801 external_mini_cluster.cc:1428] Setting key 3be8c9c286e7cd6f1a036b86bde7f605
I20260808 10:39:27.850291 2441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b: Bootstrap starting.
Aug 08 10:39:27 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:27.852931 2441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:27.853905 2441 log.cc:824] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:27.856381 2441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b: No bootstrap required, opened a new log
I20260808 10:39:27.858377 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:27.845056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.1.194.126:44449)
0808 10:39:27.845457 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:27.845471 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:27.845563 (+ 92us) server_negotiation.cc:407] Connection header received
0808 10:39:27.846203 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:27.846218 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:27.846520 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:27.846840 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:27.847636 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:27.848604 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:27.849436 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:27.849951 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:27.850124 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:27.853022 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:27.853049 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:27.853072 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:27.853112 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:27.855064 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:27.855680 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:27.855685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:27.855689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:27.855805 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:27.856128 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:27.856130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:27.856131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:27.856551 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:27.856683 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:27.857024 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 10:39:27.857326 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":138,"threads_started":1}
I20260808 10:39:27.859958 2441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377fef551f5640a0af45ae31ee3d0d2b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44449 } }
I20260808 10:39:27.860224 2441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:27.860296 2441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 377fef551f5640a0af45ae31ee3d0d2b, State: Initialized, Role: FOLLOWER
I20260808 10:39:27.860772 2441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377fef551f5640a0af45ae31ee3d0d2b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44449 } }
I20260808 10:39:27.860932 2441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:27.861032 2441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:27.861166 2441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:27.862213 2441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377fef551f5640a0af45ae31ee3d0d2b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44449 } }
I20260808 10:39:27.862560 2441 leader_election.cc:302] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 377fef551f5640a0af45ae31ee3d0d2b; no voters:
I20260808 10:39:27.862950 2441 leader_election.cc:288] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:27.863101 2447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:27.863319 2447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [term 1 LEADER]: Becoming Leader. State: Replica: 377fef551f5640a0af45ae31ee3d0d2b, State: Running, Role: LEADER
I20260808 10:39:27.863629 2447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377fef551f5640a0af45ae31ee3d0d2b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44449 } }
I20260808 10:39:27.864282 2441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:27.865753 2449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "377fef551f5640a0af45ae31ee3d0d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377fef551f5640a0af45ae31ee3d0d2b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44449 } } }
I20260808 10:39:27.865903 2449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:27.866195 2448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 377fef551f5640a0af45ae31ee3d0d2b. Latest consensus state: current_term: 1 leader_uuid: "377fef551f5640a0af45ae31ee3d0d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377fef551f5640a0af45ae31ee3d0d2b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44449 } } }
I20260808 10:39:27.866305 2448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:27.866410 2456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:27.869722 2456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:27.876252 2456 catalog_manager.cc:1347] Generated new cluster ID: fa6378b344e54c19be653a48f8fe9c14
I20260808 10:39:27.876397 2456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:27.887141 2456 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:27.888021 2456 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:27.896559 2456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 377fef551f5640a0af45ae31ee3d0d2b: Generated new TSK 0
I20260808 10:39:27.897301 2456 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:27.936307 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44449
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:28.068739 2469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:28.069001 2469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:28.069044 2469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:28.072921 2469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:28.072978 2469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:28.073036 2469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:28.078416 2469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44449
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:28.079384 2469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:28.080303 2469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:28.089710 2477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:28.089732 2474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:28.089794 2469 server_base.cc:1034] running on GCE node
W20260808 10:39:28.089717 2475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:28.090293 2469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:28.090950 2469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:28.092136 2469 hybrid_clock.cc:648] HybridClock initialized: now 1786185568092108 us; error 43 us; skew 500 ppm
Aug 08 10:39:28 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:28.095067 2469 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:28.095892 2469 webserver.cc:460] Webserver started at http://127.1.194.65:46821/ using document root <none> and password file <none>
I20260808 10:39:28.096446 2469 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:28.096536 2469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:28.096819 2469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:28.098870 2469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "0511e91bff8c46a28bc04374c2eeeafb"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "169a8310b33264e78d20f88bb1e2db23"
server_key_iv: "0228393e447403c60e13ac03c80a17f9"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.099431 2469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "0511e91bff8c46a28bc04374c2eeeafb"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "169a8310b33264e78d20f88bb1e2db23"
server_key_iv: "0228393e447403c60e13ac03c80a17f9"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.103051 2469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:39:28.105655 2484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.106808 2469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:28.106940 2469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "0511e91bff8c46a28bc04374c2eeeafb"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "169a8310b33264e78d20f88bb1e2db23"
server_key_iv: "0228393e447403c60e13ac03c80a17f9"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.107049 2469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:28.128291 2469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:28.131460 2469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:28.131678 2469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:28.132282 2469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:28.133100 2469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:28.133176 2469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.133247 2469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:28.133282 2469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.142407 2469 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:45937
I20260808 10:39:28.142434 2603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:45937 every 8 connection(s)
I20260808 10:39:28.143294 2469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:28.144652 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2469
I20260808 10:39:28.144778 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:28.145089 1801 external_mini_cluster.cc:1428] Setting key 3cb0a93a99184ecda70ad2a19bc8f109
Aug 08 10:39:28 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:39:28.158015 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.145319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:34417 (local address 127.1.194.126:44449)
0808 10:39:28.145500 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.145504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.146360 (+ 856us) server_negotiation.cc:407] Connection header received
0808 10:39:28.147793 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.147797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.147848 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.147927 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:28.149094 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.149704 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.150824 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.151074 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.151186 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.154198 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:28.154201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:28.154203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:28.154225 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:28.155764 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.156361 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.156365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.156367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.156536 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.156893 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.156895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.156897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.157077 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:28.157156 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.157690 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.157835 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:39:28.158928 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.145746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44449 (local address 127.1.194.65:34417)
0808 10:39:28.146214 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.146256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.147587 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.148104 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.148111 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.148477 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:28.148942 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.148955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.149829 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.149832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.150692 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.150702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.151226 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.151230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.151399 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.152074 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:28.152090 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:28.154004 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:28.155877 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:28.155881 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:28.155893 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:28.156235 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:28.156663 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:28.156665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:28.156667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:28.156804 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:28.157171 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:28.157176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:28.157394 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.157623 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.157877 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":145,"threads_started":1}
I20260808 10:39:28.160148 2604 heartbeater.cc:347] Connected to a master server at 127.1.194.126:44449
I20260808 10:39:28.160414 2604 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:28.161039 2604 heartbeater.cc:511] Master 127.1.194.126:44449 requested a full tablet report, sending...
I20260808 10:39:28.162889 2403 ts_manager.cc:194] Registered new tserver with Master: 0511e91bff8c46a28bc04374c2eeeafb (127.1.194.65:45937)
I20260808 10:39:28.164067 2403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:34417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:28.177673 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44449
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:28.286729 2609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:28.286968 2609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:28.287011 2609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:28.290990 2609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:28.291047 2609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:28.291146 2609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:28.296414 2609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44449
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:28.297456 2609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:28.298478 2609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:28.308223 2614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:28.308252 2617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:28.308305 2615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:28.308615 2609 server_base.cc:1034] running on GCE node
I20260808 10:39:28.308950 2609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:28.309597 2609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:28.310766 2609 hybrid_clock.cc:648] HybridClock initialized: now 1786185568310741 us; error 38 us; skew 500 ppm
Aug 08 10:39:28 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:28.313838 2609 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:28.314718 2609 webserver.cc:460] Webserver started at http://127.1.194.66:33509/ using document root <none> and password file <none>
I20260808 10:39:28.315301 2609 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:28.315363 2609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:28.315585 2609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:28.317699 2609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "f13c47ec60014a87ac3bbab12aecd670"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "31bebb129339256cafc19957fad00338"
server_key_iv: "5d322f446dcef234c5331c0acc5ff571"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.318265 2609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "f13c47ec60014a87ac3bbab12aecd670"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "31bebb129339256cafc19957fad00338"
server_key_iv: "5d322f446dcef234c5331c0acc5ff571"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.322080 2609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:39:28.324576 2624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.325946 2609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:28.326105 2609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "f13c47ec60014a87ac3bbab12aecd670"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "31bebb129339256cafc19957fad00338"
server_key_iv: "5d322f446dcef234c5331c0acc5ff571"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.326222 2609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:28.354833 2609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:28.358381 2609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:28.358628 2609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:28.359285 2609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:28.360184 2609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:28.360262 2609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.360343 2609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:28.360404 2609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.371001 2609 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:42831
I20260808 10:39:28.371021 2742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:42831 every 8 connection(s)
I20260808 10:39:28.372033 2609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:28.374745 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2609
I20260808 10:39:28.374838 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:28.375123 1801 external_mini_cluster.cc:1428] Setting key 1b949138b9130f4685ebb37dd0fa2912
Aug 08 10:39:28 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:28.387791 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.374288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:39505 (local address 127.1.194.126:44449)
0808 10:39:28.374562 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.374571 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.375432 (+ 861us) server_negotiation.cc:407] Connection header received
0808 10:39:28.376794 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.376798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.376853 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.376949 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:28.378199 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.378903 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.380785 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.381004 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.381118 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.384134 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:28.384142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:28.384144 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:28.384167 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:28.385709 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.386233 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.386239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.386241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.386322 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.386684 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.386689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.386691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.386949 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:28.387062 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.387565 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.387670 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:39:28.388664 2746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.374749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44449 (local address 127.1.194.66:39505)
0808 10:39:28.375256 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.375309 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.376594 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.377121 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.377128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.377509 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:28.378055 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.378069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.379056 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.379061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.380564 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.380579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.381177 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.381183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.381422 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.382100 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:28.382126 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:28.383954 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:28.385826 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:28.385832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:28.385847 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:28.386128 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:28.386424 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:28.386426 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:28.386429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:28.386585 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:28.387071 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:28.387076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:28.387307 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.387475 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.387727 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:28.389865 2743 heartbeater.cc:347] Connected to a master server at 127.1.194.126:44449
I20260808 10:39:28.390127 2743 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:28.390731 2743 heartbeater.cc:511] Master 127.1.194.126:44449 requested a full tablet report, sending...
I20260808 10:39:28.391842 2403 ts_manager.cc:194] Registered new tserver with Master: f13c47ec60014a87ac3bbab12aecd670 (127.1.194.66:42831)
I20260808 10:39:28.392407 2403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:39505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:28.404989 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44449
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:28.524513 2749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:28.524834 2749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:28.524915 2749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:28.529142 2749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:28.529228 2749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:28.529325 2749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:28.535053 2749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44449
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:28.536075 2749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:28.537030 2749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:39:28.547226 2749 server_base.cc:1034] running on GCE node
W20260808 10:39:28.547188 2754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:28.547199 2755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:28.547175 2757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:28.548038 2749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:28.548702 2749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:28.549890 2749 hybrid_clock.cc:648] HybridClock initialized: now 1786185568549866 us; error 36 us; skew 500 ppm
Aug 08 10:39:28 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:28.552901 2749 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:28.553864 2749 webserver.cc:460] Webserver started at http://127.1.194.67:43709/ using document root <none> and password file <none>
I20260808 10:39:28.554428 2749 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:28.554539 2749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:28.554778 2749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:28.556825 2749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "763caac8286142c7a0d90b63c94159bc"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "b6f41aa27cb104c63dbd141c1740a7b4"
server_key_iv: "1416d339fdbbea1114006c6741445bab"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.557400 2749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "763caac8286142c7a0d90b63c94159bc"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "b6f41aa27cb104c63dbd141c1740a7b4"
server_key_iv: "1416d339fdbbea1114006c6741445bab"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.561233 2749 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:39:28.564023 2764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.565438 2749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:28.565646 2749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "763caac8286142c7a0d90b63c94159bc"
format_stamp: "Formatted at 2026-08-08 10:39:28 on dist-test-slave-ss4n"
server_key: "b6f41aa27cb104c63dbd141c1740a7b4"
server_key_iv: "1416d339fdbbea1114006c6741445bab"
server_key_version: "encryptionkey@0"
I20260808 10:39:28.565750 2749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:28.582615 2749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:28.586007 2749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:28.586161 2749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:28.586753 2749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:28.587594 2749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:28.587641 2749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.587687 2749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:28.587702 2749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:28.597815 2749 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:42605
I20260808 10:39:28.597903 2881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:42605 every 8 connection(s)
I20260808 10:39:28.598745 2749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:28.602367 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2749
I20260808 10:39:28.602553 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:28.602825 1801 external_mini_cluster.cc:1428] Setting key 9cde3088569b2eec17973e363d6a8d9e
Aug 08 10:39:28 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:28.612162 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.600501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:50361 (local address 127.1.194.126:44449)
0808 10:39:28.600649 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.600653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.601446 (+ 793us) server_negotiation.cc:407] Connection header received
0808 10:39:28.602923 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.602926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.602990 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.603097 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:28.604234 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.604766 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.605962 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.606182 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.606292 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.608677 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:28.608680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:28.608682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:28.608705 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:28.610143 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.610630 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.610635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.610636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.610716 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.611117 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.611122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.611124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.611306 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:28.611414 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.611878 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.611999 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:39:28.612993 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.600865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44449 (local address 127.1.194.67:50361)
0808 10:39:28.601298 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.601339 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.602706 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.603284 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.603294 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.603639 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:28.604092 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.604103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.604892 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.604902 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.605836 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.605845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.606327 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.606331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.606494 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.607019 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:28.607034 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:28.608497 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:28.610255 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:28.610261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:28.610274 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:28.610526 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:28.610809 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:28.610813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:28.610815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:28.610991 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:28.611412 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:28.611418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:28.611652 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.611836 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.612149 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":121,"threads_started":1}
I20260808 10:39:28.614253 2882 heartbeater.cc:347] Connected to a master server at 127.1.194.126:44449
I20260808 10:39:28.614485 2882 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:28.614972 2882 heartbeater.cc:511] Master 127.1.194.126:44449 requested a full tablet report, sending...
I20260808 10:39:28.616068 2403 ts_manager.cc:194] Registered new tserver with Master: 763caac8286142c7a0d90b63c94159bc (127.1.194.67:42605)
I20260808 10:39:28.616724 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:39:28.616748 2403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:50361
I20260808 10:39:28.628656 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.621233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.1.194.126:44449)
0808 10:39:28.621357 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.621361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.621640 (+ 279us) server_negotiation.cc:407] Connection header received
0808 10:39:28.621763 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.621765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.621802 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.621871 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:28.622604 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.623093 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.624021 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.624231 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.624309 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.625754 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:28.625756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:28.625758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:28.625777 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:28.627120 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.627499 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.627502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.627503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.627597 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:28.627872 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:28.627874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:28.627875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:28.628076 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:28.628179 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.628422 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.628512 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:39:28.634385 2403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:28.637815 2403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:28.654954 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.648742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.1.194.65:45937)
0808 10:39:28.649261 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.649278 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.649323 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:39:28.649551 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.649560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.649786 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.649956 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:28.650493 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.651886 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.653473 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.654206 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.654315 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.654515 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.654677 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.654773 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":67,"threads_started":1}
I20260808 10:39:28.654943 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.648199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:45937 (local address 127.0.0.1:60242)
0808 10:39:28.649098 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.649157 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.649299 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.650158 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.650164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.650187 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:28.650343 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.650354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.652068 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.652074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.653347 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.653356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.654347 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.654351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.654466 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.654509 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.654633 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.654763 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":60,"thread_start_us":99,"threads_started":1}
I20260808 10:39:28.655596 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.649009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:42605 (local address 127.0.0.1:49504)
0808 10:39:28.650205 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.650220 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.650321 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.650875 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.650878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.650893 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:28.650971 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.650975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.652399 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.652402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.653863 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.653877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.655153 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.655157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.655270 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.655292 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.655365 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.655439 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260808 10:39:28.655757 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.648824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.1.194.66:42831)
0808 10:39:28.650225 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.650234 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.650253 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:39:28.650347 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.650354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.650529 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.650690 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:28.651311 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.652437 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.654006 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.654814 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.654915 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.655051 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.655519 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.655621 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":139,"threads_started":1}
I20260808 10:39:28.655982 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.648657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:42831 (local address 127.0.0.1:32910)
0808 10:39:28.649536 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.649551 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.649790 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.650952 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.650956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.650977 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:28.651176 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.651183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.652574 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.652578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.653883 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.653890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.655123 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.655128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.655244 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.655265 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.655333 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.655411 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":28,"thread_start_us":130,"threads_started":1}
I20260808 10:39:28.658646 2534 tablet_service.cc:1467] Processing CreateTablet for tablet 05670c4a69e5499b8af86c6e19a34c22 (DEFAULT_TABLE table=test-table [id=b282e6b775eb481087a3b381e14a3b39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:28.655997 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.649418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.1.194.67:42605)
0808 10:39:28.649895 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.649907 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.650328 (+ 421us) server_negotiation.cc:407] Connection header received
0808 10:39:28.650412 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.650417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.650586 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.650716 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:28.651096 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.652247 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.654056 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.654906 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.655026 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.655194 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.655514 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.655614 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":190,"threads_started":1}
I20260808 10:39:28.659922 2534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05670c4a69e5499b8af86c6e19a34c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:28.660758 2672 tablet_service.cc:1467] Processing CreateTablet for tablet 05670c4a69e5499b8af86c6e19a34c22 (DEFAULT_TABLE table=test-table [id=b282e6b775eb481087a3b381e14a3b39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:28.661767 2672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05670c4a69e5499b8af86c6e19a34c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:28.661803 2813 tablet_service.cc:1467] Processing CreateTablet for tablet 05670c4a69e5499b8af86c6e19a34c22 (DEFAULT_TABLE table=test-table [id=b282e6b775eb481087a3b381e14a3b39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:28.662873 2813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05670c4a69e5499b8af86c6e19a34c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:28.668351 2901 tablet_bootstrap.cc:492] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: Bootstrap starting.
I20260808 10:39:28.669143 2902 tablet_bootstrap.cc:492] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: Bootstrap starting.
I20260808 10:39:28.670699 2903 tablet_bootstrap.cc:492] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: Bootstrap starting.
I20260808 10:39:28.671613 2902 tablet_bootstrap.cc:654] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:28.672313 2901 tablet_bootstrap.cc:654] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:28.673086 2902 log.cc:824] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:28.673352 2901 log.cc:824] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:28.674149 2903 tablet_bootstrap.cc:654] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:28.674926 2903 log.cc:824] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:28.675324 2902 tablet_bootstrap.cc:492] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: No bootstrap required, opened a new log
I20260808 10:39:28.675535 2902 ts_tablet_manager.cc:1397] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260808 10:39:28.676720 2901 tablet_bootstrap.cc:492] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: No bootstrap required, opened a new log
I20260808 10:39:28.676841 2903 tablet_bootstrap.cc:492] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: No bootstrap required, opened a new log
I20260808 10:39:28.676965 2901 ts_tablet_manager.cc:1397] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260808 10:39:28.677070 2903 ts_tablet_manager.cc:1397] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:39:28.679911 2902 raft_consensus.cc:348] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.680189 2902 raft_consensus.cc:374] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:28.680265 2902 raft_consensus.cc:729] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0511e91bff8c46a28bc04374c2eeeafb, State: Initialized, Role: FOLLOWER
I20260808 10:39:28.680743 2902 consensus_queue.cc:260] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.681566 2604 heartbeater.cc:503] Master 127.1.194.126:44449 was elected leader, sending a full tablet report...
I20260808 10:39:28.681905 2902 ts_tablet_manager.cc:1428] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 10:39:28.681825 2901 raft_consensus.cc:348] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.681825 2903 raft_consensus.cc:348] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.682132 2901 raft_consensus.cc:374] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:28.682192 2903 raft_consensus.cc:374] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:28.682225 2901 raft_consensus.cc:729] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13c47ec60014a87ac3bbab12aecd670, State: Initialized, Role: FOLLOWER
I20260808 10:39:28.682251 2903 raft_consensus.cc:729] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763caac8286142c7a0d90b63c94159bc, State: Initialized, Role: FOLLOWER
I20260808 10:39:28.682673 2903 consensus_queue.cc:260] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.682766 2901 consensus_queue.cc:260] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.683746 2743 heartbeater.cc:503] Master 127.1.194.126:44449 was elected leader, sending a full tablet report...
I20260808 10:39:28.683806 2901 ts_tablet_manager.cc:1428] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:39:28.683806 2903 ts_tablet_manager.cc:1428] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:39:28.684224 2882 heartbeater.cc:503] Master 127.1.194.126:44449 was elected leader, sending a full tablet report...
W20260808 10:39:28.849772 2884 tablet.cc:2369] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:28.873656 2745 tablet.cc:2369] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:28.894754 2605 tablet.cc:2369] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:28.988938 2908 raft_consensus.cc:482] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:28.989219 2908 raft_consensus.cc:504] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.990533 2908 leader_election.cc:288] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0511e91bff8c46a28bc04374c2eeeafb (127.1.194.65:45937), 763caac8286142c7a0d90b63c94159bc (127.1.194.67:42605)
I20260808 10:39:28.995366 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.990795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:45937 (local address 127.1.194.66:49513)
0808 10:39:28.991088 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.991105 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.991216 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.991577 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.991580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.991604 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:28.991726 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.991731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.992805 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.992808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.993919 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.993929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.994828 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.994832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.994935 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.994962 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.995046 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.995135 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
I20260808 10:39:28.995426 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.990975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:49513 (local address 127.1.194.65:45937)
0808 10:39:28.991129 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.991133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.991156 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:39:28.991354 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.991357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.991422 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.991547 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:28.991855 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.992682 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.994153 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.994705 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.994799 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.994839 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.995098 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.995295 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:39:28.995764 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.991496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:57207 (local address 127.1.194.67:42605)
0808 10:39:28.991681 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:28.991686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:28.991710 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:28.991929 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:28.991932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:28.991973 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:28.992032 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:28.992357 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.992923 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.994506 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.995070 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.995180 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.995225 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:28.995545 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 10:39:28.995632 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:39:28.996167 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:28.991238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:42605 (local address 127.1.194.66:57207)
0808 10:39:28.991479 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:28.991496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:28.991771 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:28.992077 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:28.992080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:28.992100 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:28.992226 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.992242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.993039 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.993042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.994376 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:28.994386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:28.995283 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:28.995287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:28.995386 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:28.995408 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:39:28.995910 (+ 502us) client_negotiation.cc:240] Negotiation successful
0808 10:39:28.995983 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260808 10:39:28.996268 2556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05670c4a69e5499b8af86c6e19a34c22" candidate_uuid: "f13c47ec60014a87ac3bbab12aecd670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0511e91bff8c46a28bc04374c2eeeafb" is_pre_election: true
I20260808 10:39:28.996727 2556 raft_consensus.cc:2454] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13c47ec60014a87ac3bbab12aecd670 in term 0.
I20260808 10:39:28.996831 2836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05670c4a69e5499b8af86c6e19a34c22" candidate_uuid: "f13c47ec60014a87ac3bbab12aecd670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "763caac8286142c7a0d90b63c94159bc" is_pre_election: true
I20260808 10:39:28.997159 2836 raft_consensus.cc:2454] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13c47ec60014a87ac3bbab12aecd670 in term 0.
I20260808 10:39:28.997377 2626 leader_election.cc:302] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0511e91bff8c46a28bc04374c2eeeafb, f13c47ec60014a87ac3bbab12aecd670; no voters:
I20260808 10:39:28.997737 2908 raft_consensus.cc:2781] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:28.997833 2908 raft_consensus.cc:482] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:28.997895 2908 raft_consensus.cc:3037] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:28.999054 2908 raft_consensus.cc:504] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:28.999399 2908 leader_election.cc:288] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [CANDIDATE]: Term 1 election: Requested vote from peers 0511e91bff8c46a28bc04374c2eeeafb (127.1.194.65:45937), 763caac8286142c7a0d90b63c94159bc (127.1.194.67:42605)
I20260808 10:39:28.999814 2836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05670c4a69e5499b8af86c6e19a34c22" candidate_uuid: "f13c47ec60014a87ac3bbab12aecd670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "763caac8286142c7a0d90b63c94159bc"
I20260808 10:39:28.999840 2556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05670c4a69e5499b8af86c6e19a34c22" candidate_uuid: "f13c47ec60014a87ac3bbab12aecd670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0511e91bff8c46a28bc04374c2eeeafb"
I20260808 10:39:28.999949 2836 raft_consensus.cc:3037] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:28.999976 2556 raft_consensus.cc:3037] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:29.001168 2556 raft_consensus.cc:2454] T 05670c4a69e5499b8af86c6e19a34c22 P 0511e91bff8c46a28bc04374c2eeeafb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13c47ec60014a87ac3bbab12aecd670 in term 1.
I20260808 10:39:29.001169 2836 raft_consensus.cc:2454] T 05670c4a69e5499b8af86c6e19a34c22 P 763caac8286142c7a0d90b63c94159bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13c47ec60014a87ac3bbab12aecd670 in term 1.
I20260808 10:39:29.001524 2626 leader_election.cc:302] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0511e91bff8c46a28bc04374c2eeeafb, f13c47ec60014a87ac3bbab12aecd670; no voters:
I20260808 10:39:29.001725 2908 raft_consensus.cc:2781] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:29.001966 2908 raft_consensus.cc:686] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [term 1 LEADER]: Becoming Leader. State: Replica: f13c47ec60014a87ac3bbab12aecd670, State: Running, Role: LEADER
I20260808 10:39:29.002331 2908 consensus_queue.cc:237] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } }
I20260808 10:39:29.005846 2402 catalog_manager.cc:5495] T 05670c4a69e5499b8af86c6e19a34c22 P f13c47ec60014a87ac3bbab12aecd670 reported cstate change: term changed from 0 to 1, leader changed from <none> to f13c47ec60014a87ac3bbab12aecd670 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "f13c47ec60014a87ac3bbab12aecd670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511e91bff8c46a28bc04374c2eeeafb" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "763caac8286142c7a0d90b63c94159bc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f13c47ec60014a87ac3bbab12aecd670" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42831 } health_report { overall_health: HEALTHY } } }
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:29.022891 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:29.012610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.1.194.65:45937)
0808 10:39:29.012822 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:29.012827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:29.013065 (+ 238us) server_negotiation.cc:407] Connection header received
0808 10:39:29.013249 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:29.013251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:29.013295 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:29.013379 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:29.014153 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:29.014676 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:29.015685 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:29.015910 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:29.016019 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:29.018938 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:29.018951 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:29.018967 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:29.018992 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:29.021090 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:29.021646 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:29.021652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:29.021653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:29.021732 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:29.021972 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:29.021974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:29.021975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:29.022242 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:29.022389 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:29.022618 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:39:29.022717 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185569, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185569, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:29.053869 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:29.044194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.1.194.65:45937)
0808 10:39:29.044377 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:29.044383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:29.044491 (+ 108us) server_negotiation.cc:407] Connection header received
0808 10:39:29.044672 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:29.044676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:29.044728 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:29.044820 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:29.045672 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:29.046290 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:29.047147 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:29.047431 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:29.047555 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:29.050279 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:29.050287 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:29.050289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:29.050314 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:29.051943 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:29.052394 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:29.052403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:29.052404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:29.052480 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:29.052764 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:29.052768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:29.052769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:29.052940 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:29.053199 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:29.053561 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 10:39:29.053698 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:29.082068 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:29.072832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.1.194.65:45937)
0808 10:39:29.073021 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:29.073026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:29.073109 (+ 83us) server_negotiation.cc:407] Connection header received
0808 10:39:29.073232 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:29.073235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:29.073279 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:29.073351 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:29.074223 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:29.074790 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:29.075741 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:29.076011 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:29.076115 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:29.078900 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:29.078903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:29.078906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:29.078929 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:29.080423 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:29.080855 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:29.080858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:29.080859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:29.080929 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:29.081157 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:29.081160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:29.081161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:29.081315 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:29.081409 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:29.081739 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 10:39:29.081843 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260808 10:39:29.083760 2514 tablet_service.cc:2908] Rejecting scan request for tablet 05670c4a69e5499b8af86c6e19a34c22: Uninitialized: clean time has not yet been initialized
I20260808 10:39:29.086035 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2469
I20260808 10:39:29.093164 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2609
I20260808 10:39:29.100844 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2749
I20260808 10:39:29.107103 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2371
2026-08-08T10:39:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2669 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2928](info): set up 2 sockets
Aug 08 10:39:29 dist-test-slave-ss4n krb5kdc[2928](info): commencing operation
krb5kdc: starting...
W20260808 10:39:30.188510 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:30 dist-test-slave-ss4n krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:30.321997 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:40311
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:40311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:30.438230 2942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:30.438503 2942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:30.438594 2942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:30.442821 2942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:30.442881 2942 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:39:30.442898 2942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:30.442916 2942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:30.442934 2942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:30.442950 2942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:30.448200 2942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:40311
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:40311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:30.449357 2942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:30.450393 2942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:30.459071 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:30.459112 2948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:30.459182 2942 server_base.cc:1034] running on GCE node
W20260808 10:39:30.459112 2947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:30.459863 2942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:30.461014 2942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:30.462219 2942 hybrid_clock.cc:648] HybridClock initialized: now 1786185570462197 us; error 35 us; skew 500 ppm
Aug 08 10:39:30 dist-test-slave-ss4n krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:30.465101 2942 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:30.465983 2942 webserver.cc:460] Webserver started at http://127.1.194.126:40725/ using document root <none> and password file <none>
I20260808 10:39:30.466498 2942 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:30.466557 2942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:30.466717 2942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:30.468683 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "78a695d3be4d45009324a708931fa2f3"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "7242b2e0399823cfda5e0ba3a9a4d8ce"
server_key_iv: "7ffee23dd4095d24a52a2d1afb674d14"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.469281 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "78a695d3be4d45009324a708931fa2f3"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "7242b2e0399823cfda5e0ba3a9a4d8ce"
server_key_iv: "7ffee23dd4095d24a52a2d1afb674d14"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.473001 2942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:30.475594 2959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:30.476881 2942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:30.476994 2942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "78a695d3be4d45009324a708931fa2f3"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "7242b2e0399823cfda5e0ba3a9a4d8ce"
server_key_iv: "7ffee23dd4095d24a52a2d1afb674d14"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.477097 2942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:30.498502 2942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:30.501720 2942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:30.501876 2942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:30.510322 2942 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:40311
I20260808 10:39:30.510347 3024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:40311 every 8 connection(s)
I20260808 10:39:30.511253 2942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:30.513986 3025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:30.519351 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: Bootstrap starting.
I20260808 10:39:30.520072 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 2942
I20260808 10:39:30.520174 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:30.520455 1801 external_mini_cluster.cc:1428] Setting key 586898ca13b209e5f0742189838ef2e4
I20260808 10:39:30.522095 3025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:30.522830 3025 log.cc:824] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:30.524644 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: No bootstrap required, opened a new log
I20260808 10:39:30.528331 3025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a695d3be4d45009324a708931fa2f3" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 40311 } }
I20260808 10:39:30.528654 3025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:30.528743 3025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78a695d3be4d45009324a708931fa2f3, State: Initialized, Role: FOLLOWER
Aug 08 10:39:30 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:30.529286 3025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a695d3be4d45009324a708931fa2f3" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 40311 } }
I20260808 10:39:30.529454 3025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:30.529556 3025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:30.529659 3025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:30.530882 3025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a695d3be4d45009324a708931fa2f3" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 40311 } }
I20260808 10:39:30.531237 3025 leader_election.cc:302] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78a695d3be4d45009324a708931fa2f3; no voters:
I20260808 10:39:30.531579 3025 leader_election.cc:288] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:30.531685 3030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:30.531919 3030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [term 1 LEADER]: Becoming Leader. State: Replica: 78a695d3be4d45009324a708931fa2f3, State: Running, Role: LEADER
I20260808 10:39:30.532266 3030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a695d3be4d45009324a708931fa2f3" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 40311 } }
I20260808 10:39:30.532869 3025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:30.534034 3032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78a695d3be4d45009324a708931fa2f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a695d3be4d45009324a708931fa2f3" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 40311 } } }
I20260808 10:39:30.534075 3033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78a695d3be4d45009324a708931fa2f3. Latest consensus state: current_term: 1 leader_uuid: "78a695d3be4d45009324a708931fa2f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78a695d3be4d45009324a708931fa2f3" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 40311 } } }
I20260808 10:39:30.534160 3032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:30.534183 3033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:30.534500 3040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:30.535306 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:30.521623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.1.194.126:40311)
0808 10:39:30.522032 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:30.522049 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:30.522090 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:39:30.522770 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:30.522788 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:30.523148 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:30.523466 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:30.524265 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.525274 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.526181 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.526643 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.526810 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:30.529641 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:30.529649 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:30.529665 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:30.529701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:30.531609 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:30.532207 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:30.532212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:30.532218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:30.532375 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:30.532707 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:30.532711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:30.532713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:30.533167 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:30.533377 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:30.534070 (+ 693us) server_negotiation.cc:299] Negotiation successful
0808 10:39:30.534380 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":133,"threads_started":1}
I20260808 10:39:30.538160 3040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:30.543713 3040 catalog_manager.cc:1347] Generated new cluster ID: 94ce0e0c39f44c25b3978cce162f47ea
I20260808 10:39:30.543793 3040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:30.558732 3040 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:30.560063 3040 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:30.568099 3040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: Generated new TSK 0
I20260808 10:39:30.568988 3040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:30.605417 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:40311
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:30.720695 3052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:30.720980 3052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:30.721083 3052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:30.725627 3052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:30.725698 3052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:30.725742 3052 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:39:30.725780 3052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:30.731805 3052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:40311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:30.733000 3052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:30.734223 3052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:30.744428 3058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:30.744441 3060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:30.744479 3057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:30.746107 3052 server_base.cc:1034] running on GCE node
I20260808 10:39:30.746588 3052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:30.747283 3052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:30.748489 3052 hybrid_clock.cc:648] HybridClock initialized: now 1786185570748461 us; error 45 us; skew 500 ppm
Aug 08 10:39:30 dist-test-slave-ss4n krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:30.751719 3052 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:30.752701 3052 webserver.cc:460] Webserver started at http://127.1.194.65:35603/ using document root <none> and password file <none>
I20260808 10:39:30.753327 3052 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:30.753425 3052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:30.753708 3052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:30.755911 3052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "f1acab4297ce4a69ab5eb413a20c8a80"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "5739ad270d18fe20d6cd8fef3b081e4f"
server_key_iv: "b1c7e30efd9fb32e0fc2f440455fbc21"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.756632 3052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "f1acab4297ce4a69ab5eb413a20c8a80"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "5739ad270d18fe20d6cd8fef3b081e4f"
server_key_iv: "b1c7e30efd9fb32e0fc2f440455fbc21"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.760911 3052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:39:30.763777 3067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:30.765162 3052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:39:30.765326 3052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "f1acab4297ce4a69ab5eb413a20c8a80"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "5739ad270d18fe20d6cd8fef3b081e4f"
server_key_iv: "b1c7e30efd9fb32e0fc2f440455fbc21"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.765456 3052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:30.776131 3052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:30.779753 3052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:30.779969 3052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:30.781839 3052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:30.781905 3052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:30.781961 3052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:30.781996 3052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:39:30 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:30.794608 3052 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:39291
I20260808 10:39:30.794715 3191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:39291 every 8 connection(s)
I20260808 10:39:30.795581 3052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:30.796936 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:30.782655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:45295 (local address 127.1.194.126:40311)
0808 10:39:30.782769 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:30.782773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:30.783872 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 10:39:30.784936 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:30.784943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:30.785004 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:30.785085 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:30.786544 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.787402 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.788695 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.788944 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.789060 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:30.793110 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:30.793114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:30.793116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:30.793144 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:30.794872 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:30.795376 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:30.795379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:30.795381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:30.795472 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:30.795836 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:30.795840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:30.795842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:30.796034 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:30.796159 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:30.796695 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 10:39:30.796810 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:39:30.797725 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:30.783237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.1.194.65:45295)
0808 10:39:30.783730 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:30.783771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:30.784694 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:30.785571 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:30.785585 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:30.785887 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:30.786397 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:30.786411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.787521 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.787525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:30.788576 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:30.788585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.789103 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.789108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:30.789267 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:30.789946 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:30.789974 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:30.792939 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:30.794994 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:30.795000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:30.795014 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:30.795262 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:30.795570 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:30.795573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:30.795575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:30.795748 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:30.796210 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:30.796216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:30.796482 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 10:39:30.796635 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 10:39:30.796862 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260808 10:39:30.799098 3192 heartbeater.cc:347] Connected to a master server at 127.1.194.126:40311
I20260808 10:39:30.799269 3192 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:30.799760 3192 heartbeater.cc:511] Master 127.1.194.126:40311 requested a full tablet report, sending...
I20260808 10:39:30.801366 2972 ts_manager.cc:194] Registered new tserver with Master: f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291)
I20260808 10:39:30.802822 2972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:45295
I20260808 10:39:30.802882 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 3052
I20260808 10:39:30.803017 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:30.803318 1801 external_mini_cluster.cc:1428] Setting key 7d13870d2732d40afce7a5c511223465
I20260808 10:39:30.810310 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:30.801355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:60703 (local address 127.1.194.126:40311)
0808 10:39:30.801514 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:30.801519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:30.801776 (+ 257us) server_negotiation.cc:407] Connection header received
0808 10:39:30.801873 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:30.801875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:30.801919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:30.801992 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:30.802578 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.803447 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.804905 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.805173 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.805255 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:30.806699 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:30.806702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:30.806704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:30.806725 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:30.808659 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:30.809101 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:30.809105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:30.809107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:30.809188 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:30.809471 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:30.809476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:30.809478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:30.809674 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:30.809779 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:30.810031 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 10:39:30.810138 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:39:30.810331 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:30.801228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.1.194.65:60703)
0808 10:39:30.801685 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:30.801700 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:30.801791 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:30.802028 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:30.802031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:30.802258 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:30.802375 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:30.802382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.803730 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.803735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:30.804770 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:30.804783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:30.805321 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:30.805328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:30.805446 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:30.806000 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:30.806022 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:30.806558 (+ 536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:30.808775 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:30.808780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:30.808782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:30.808982 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:30.809289 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:30.809292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:30.809294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:30.809380 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:30.809770 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:30.809774 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:30.809871 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 10:39:30.809978 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:39:30.810139 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:30.838217 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:40311
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:30.956041 3201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:30.956288 3201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:30.956333 3201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:30.960386 3201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:30.960448 3201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:30.960489 3201 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:39:30.960521 3201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:30.965906 3201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:40311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:30.966890 3201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:30.967828 3201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:39:30.977864 3201 server_base.cc:1034] running on GCE node
W20260808 10:39:30.977824 3206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:30.977836 3209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:30.977824 3207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:30.978554 3201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:30.979233 3201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:30.980453 3201 hybrid_clock.cc:648] HybridClock initialized: now 1786185570980424 us; error 42 us; skew 500 ppm
Aug 08 10:39:30 dist-test-slave-ss4n krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:30.983665 3201 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:30.984704 3201 webserver.cc:460] Webserver started at http://127.1.194.66:42641/ using document root <none> and password file <none>
I20260808 10:39:30.985296 3201 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:30.985394 3201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:30.985742 3201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:30.987963 3201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "51fb37abead64d859f509b346f3b7028"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "b033db97814267341accf2e9834dc6b0"
server_key_iv: "7fd83af741f20ef0c7bcd98803985b6b"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.988595 3201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "51fb37abead64d859f509b346f3b7028"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "b033db97814267341accf2e9834dc6b0"
server_key_iv: "7fd83af741f20ef0c7bcd98803985b6b"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.992731 3201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:39:30.995724 3219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:30.997102 3201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:30.997255 3201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "51fb37abead64d859f509b346f3b7028"
format_stamp: "Formatted at 2026-08-08 10:39:30 on dist-test-slave-ss4n"
server_key: "b033db97814267341accf2e9834dc6b0"
server_key_iv: "7fd83af741f20ef0c7bcd98803985b6b"
server_key_version: "encryptionkey@0"
I20260808 10:39:30.997383 3201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:31.016008 3201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:31.019690 3201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:31.019910 3201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:31.021515 3201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:31.021590 3201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:31.021668 3201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:31.021710 3201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:39:31 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:31.034163 3201 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:42025
I20260808 10:39:31.035413 3201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:31.036356 3344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:42025 every 8 connection(s)
I20260808 10:39:31.037855 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.022237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:36485 (local address 127.1.194.126:40311)
0808 10:39:31.022362 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.022366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.024233 (+ 1867us) server_negotiation.cc:407] Connection header received
0808 10:39:31.025450 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.025453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.025549 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.025627 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:31.026897 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.027500 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.028757 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.029107 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.029226 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.033205 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:31.033210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:31.033213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:31.033244 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:31.035466 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.036203 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.036209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.036213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.036324 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.036743 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.036746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.036747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.036939 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:31.037023 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.037593 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.037735 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:31.038760 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.023518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.1.194.66:36485)
0808 10:39:31.024070 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.024130 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.025248 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.025788 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.025798 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.026124 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:31.026749 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.026765 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.027634 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.027638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.028621 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.028631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.029280 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.029284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.029520 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.030193 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:31.030223 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:31.032976 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:31.035618 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.035624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.035639 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.036056 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.036450 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.036453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.036456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.036637 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.037050 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:31.037055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:31.037293 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.037459 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.037836 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":152,"threads_started":1}
I20260808 10:39:31.040331 3346 heartbeater.cc:347] Connected to a master server at 127.1.194.126:40311
I20260808 10:39:31.040524 3346 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:31.041106 3346 heartbeater.cc:511] Master 127.1.194.126:40311 requested a full tablet report, sending...
I20260808 10:39:31.042352 2976 ts_manager.cc:194] Registered new tserver with Master: 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025)
I20260808 10:39:31.043159 2976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:36485
I20260808 10:39:31.045598 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 3201
I20260808 10:39:31.045722 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:31.046034 1801 external_mini_cluster.cc:1428] Setting key 9a19f1bdab684d1e30e6d8c3a967ec9a
I20260808 10:39:31.050925 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.042899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:36995 (local address 127.1.194.126:40311)
0808 10:39:31.043068 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.043073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.043306 (+ 233us) server_negotiation.cc:407] Connection header received
0808 10:39:31.043481 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.043484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.043537 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.043629 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:31.044166 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.044735 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.045919 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.046218 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.046310 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.047644 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:31.047647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:31.047650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:31.047674 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:31.049150 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.049617 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.049621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.049623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.049709 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.049985 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.050009 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.050010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.050228 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:31.050342 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.050662 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.050766 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 10:39:31.050937 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.042781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.1.194.66:36995)
0808 10:39:31.043211 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.043224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.043347 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.043647 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.043650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.043930 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:31.044045 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.044051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.044876 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.044879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.045765 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.045781 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.046346 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.046350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.046482 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.047054 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:31.047066 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:31.047509 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:31.049263 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.049268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.049270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.049531 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.049802 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.049806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.049807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.049894 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.050330 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:31.050333 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:31.050433 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.050609 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.050759 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":144,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:31.078930 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:40311
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:31.192281 3356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:31.192559 3356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:31.192605 3356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:31.196699 3356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:31.196758 3356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:31.196798 3356 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:39:31.196870 3356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:31.202569 3356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:40311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:31.203680 3356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:31.204706 3356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:31.214609 3361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:31.214618 3364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:31.214632 3362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:31.214717 3356 server_base.cc:1034] running on GCE node
I20260808 10:39:31.215610 3356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:31.216358 3356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:31.217629 3356 hybrid_clock.cc:648] HybridClock initialized: now 1786185571217611 us; error 32 us; skew 500 ppm
Aug 08 10:39:31 dist-test-slave-ss4n krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185571, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:31.220791 3356 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:31.221814 3356 webserver.cc:460] Webserver started at http://127.1.194.67:41903/ using document root <none> and password file <none>
I20260808 10:39:31.222378 3356 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:31.222441 3356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:31.222616 3356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:31.224690 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "e29e39a570ef43248ffd1eec2aa2a359"
format_stamp: "Formatted at 2026-08-08 10:39:31 on dist-test-slave-ss4n"
server_key: "75270a838a75fac7a4cb67766f7101ab"
server_key_iv: "8ab1968269664d79e1c3850e5026e99c"
server_key_version: "encryptionkey@0"
I20260808 10:39:31.225286 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "e29e39a570ef43248ffd1eec2aa2a359"
format_stamp: "Formatted at 2026-08-08 10:39:31 on dist-test-slave-ss4n"
server_key: "75270a838a75fac7a4cb67766f7101ab"
server_key_iv: "8ab1968269664d79e1c3850e5026e99c"
server_key_version: "encryptionkey@0"
I20260808 10:39:31.229226 3356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:39:31.231923 3372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:31.233290 3356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:39:31.233389 3356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "e29e39a570ef43248ffd1eec2aa2a359"
format_stamp: "Formatted at 2026-08-08 10:39:31 on dist-test-slave-ss4n"
server_key: "75270a838a75fac7a4cb67766f7101ab"
server_key_iv: "8ab1968269664d79e1c3850e5026e99c"
server_key_version: "encryptionkey@0"
I20260808 10:39:31.233615 3356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:31.247692 3356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:31.250836 3356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:31.250994 3356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:31.252388 3356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:31.252437 3356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:31.252508 3356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:31.252547 3356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:39:31 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185571, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:31.263693 3356 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:38839
I20260808 10:39:31.263756 3489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:38839 every 8 connection(s)
I20260808 10:39:31.264683 3356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:31.266521 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.253136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:44441 (local address 127.1.194.126:40311)
0808 10:39:31.253300 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.253305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.254553 (+ 1248us) server_negotiation.cc:407] Connection header received
0808 10:39:31.255712 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.255716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.255771 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.255897 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:31.257331 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.257932 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.259050 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.259399 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.259517 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.262645 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:31.262647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:31.262650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:31.262674 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:31.264344 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.264916 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.264919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.264920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.265021 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.265447 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.265450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.265452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.265693 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:31.265774 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.266201 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.266347 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:39:31.266507 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 3356
I20260808 10:39:31.266634 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:31.266899 1801 external_mini_cluster.cc:1428] Setting key 5f0d20a9a05fd0ed8ee14d5c455b2b81
I20260808 10:39:31.267431 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.253953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.1.194.67:44441)
0808 10:39:31.254382 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.254429 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.255512 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.256196 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.256202 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.256665 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:31.257185 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.257198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.258061 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.258064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.258904 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.258912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.259552 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.259556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.259746 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.260338 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:31.260363 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:31.262473 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:31.264463 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.264469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.264482 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.264823 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.265155 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.265158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.265161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.265334 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.265780 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:31.265784 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:31.265998 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.266150 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.266400 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
I20260808 10:39:31.268765 3490 heartbeater.cc:347] Connected to a master server at 127.1.194.126:40311
I20260808 10:39:31.268911 3490 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:31.269698 3490 heartbeater.cc:511] Master 127.1.194.126:40311 requested a full tablet report, sending...
I20260808 10:39:31.270699 2972 ts_manager.cc:194] Registered new tserver with Master: e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839)
I20260808 10:39:31.271555 2972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:44441
I20260808 10:39:31.277733 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.270945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:58913 (local address 127.1.194.126:40311)
0808 10:39:31.271072 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.271076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.271231 (+ 155us) server_negotiation.cc:407] Connection header received
0808 10:39:31.271345 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.271347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.271381 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.271441 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:31.271936 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.272431 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.273220 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.273447 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.273596 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.274674 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:31.274678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:31.274680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:31.274703 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:31.276120 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.276542 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.276547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.276549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.276626 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.276890 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.276895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.276897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.277077 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:31.277182 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.277431 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.277571 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:39:31.277750 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.270865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.1.194.67:58913)
0808 10:39:31.271162 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.271178 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.271270 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.271483 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.271486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.271697 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:31.271828 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.271835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.272561 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.272564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.273107 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.273114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.273601 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.273604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.273708 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.274216 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:31.274228 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:31.274543 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:31.276246 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.276250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.276252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.276440 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.276717 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.276720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.276722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.276804 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.277185 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:31.277189 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:31.277279 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.277381 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.277554 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":132,"threads_started":1}
I20260808 10:39:31.280934 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:39:31 dist-test-slave-ss4n krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185571, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:39:31 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185571, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:31.309429 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.300635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.1.194.126:40311)
0808 10:39:31.300781 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.300789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.300805 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:39:31.300862 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.300865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.300906 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.301036 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:31.302043 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.302746 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.303584 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.303817 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.303907 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.306440 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:31.306443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:31.306446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:31.306472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:31.307856 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.308256 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.308259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.308260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.308338 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.308584 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.308587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.308588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.308766 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:31.308876 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.309089 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.309190 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:39:31.312542 2972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:31.315791 2972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:31.330258 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.325211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.1.194.65:39291)
0808 10:39:31.325934 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.325942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.325963 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:39:31.326037 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.326042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.326245 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.326401 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.326784 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.327566 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.328900 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.329558 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.329678 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.329839 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.330019 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.330126 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":99,"threads_started":1}
I20260808 10:39:31.330258 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.324671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39291 (local address 127.0.0.1:53854)
0808 10:39:31.325659 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.325673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.325754 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.326420 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.326430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.326460 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.326658 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.326665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.327690 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.327696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.328764 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.328776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.329675 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.329678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.329804 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.329840 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.329975 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.330102 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":34,"thread_start_us":97,"threads_started":1}
I20260808 10:39:31.331314 3510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.324471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:38839 (local address 127.0.0.1:40572)
0808 10:39:31.325265 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.325324 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.325514 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.326492 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.326495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.326525 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.326639 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.326649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.328457 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.328460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.329737 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.329749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.330868 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.330872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.330991 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.331016 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.331092 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.331166 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":61,"spinlock_wait_cycles":8576,"thread_start_us":135,"threads_started":1}
I20260808 10:39:31.331347 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.324662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.1.194.67:38839)
0808 10:39:31.325876 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.325890 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.325914 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:31.325997 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.326004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.326310 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.326813 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.326967 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.328310 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.329891 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.330724 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.330870 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.331065 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.331123 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.331212 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1074,"thread_start_us":113,"threads_started":1}
I20260808 10:39:31.332653 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.324462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:42025 (local address 127.0.0.1:59054)
0808 10:39:31.325265 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.325350 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.325507 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.326576 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.326579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.326594 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.326805 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.326812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.328457 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.328460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.330679 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.330691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.332205 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.332209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.332320 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.332345 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.332425 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.332495 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":141,"spinlock_wait_cycles":9344,"thread_start_us":62,"threads_started":1}
I20260808 10:39:31.333014 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.325211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.1.194.66:42025)
0808 10:39:31.326008 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.326020 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.326038 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:39:31.326102 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.326106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.326244 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.326401 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.326952 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.328307 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.330836 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.332074 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.332180 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.332321 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.332624 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.332872 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":92,"threads_started":1}
I20260808 10:39:31.333947 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 325961a20bd5432b8daeffb64ac77f94 (DEFAULT_TABLE table=test-table [id=1a690423049042cb848ea11335d9fe12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:31.334257 3423 tablet_service.cc:1467] Processing CreateTablet for tablet 325961a20bd5432b8daeffb64ac77f94 (DEFAULT_TABLE table=test-table [id=1a690423049042cb848ea11335d9fe12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:31.335186 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325961a20bd5432b8daeffb64ac77f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.335356 3423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325961a20bd5432b8daeffb64ac77f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.335970 3272 tablet_service.cc:1467] Processing CreateTablet for tablet 325961a20bd5432b8daeffb64ac77f94 (DEFAULT_TABLE table=test-table [id=1a690423049042cb848ea11335d9fe12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:31.336906 3272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325961a20bd5432b8daeffb64ac77f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.343081 3514 tablet_bootstrap.cc:492] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Bootstrap starting.
I20260808 10:39:31.344360 3515 tablet_bootstrap.cc:492] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Bootstrap starting.
I20260808 10:39:31.345348 3514 tablet_bootstrap.cc:654] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.345867 3516 tablet_bootstrap.cc:492] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Bootstrap starting.
I20260808 10:39:31.346295 3515 tablet_bootstrap.cc:654] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.346447 3514 log.cc:824] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:31.347183 3515 log.cc:824] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:31.348284 3516 tablet_bootstrap.cc:654] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.349138 3516 log.cc:824] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:31.349241 3515 tablet_bootstrap.cc:492] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: No bootstrap required, opened a new log
I20260808 10:39:31.349310 3514 tablet_bootstrap.cc:492] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: No bootstrap required, opened a new log
I20260808 10:39:31.349439 3515 ts_tablet_manager.cc:1397] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:39:31.349500 3514 ts_tablet_manager.cc:1397] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:39:31.350991 3516 tablet_bootstrap.cc:492] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: No bootstrap required, opened a new log
I20260808 10:39:31.351215 3516 ts_tablet_manager.cc:1397] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 10:39:31.355405 3515 raft_consensus.cc:348] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.355695 3515 raft_consensus.cc:374] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.355756 3515 raft_consensus.cc:729] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29e39a570ef43248ffd1eec2aa2a359, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.355800 3516 raft_consensus.cc:348] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.356097 3516 raft_consensus.cc:374] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.356168 3516 raft_consensus.cc:729] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fb37abead64d859f509b346f3b7028, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.356074 3514 raft_consensus.cc:348] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.356215 3515 consensus_queue.cc:260] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.356304 3514 raft_consensus.cc:374] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.356352 3514 raft_consensus.cc:729] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1acab4297ce4a69ab5eb413a20c8a80, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.356616 3516 consensus_queue.cc:260] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.356770 3514 consensus_queue.cc:260] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.357223 3490 heartbeater.cc:503] Master 127.1.194.126:40311 was elected leader, sending a full tablet report...
I20260808 10:39:31.357365 3515 ts_tablet_manager.cc:1428] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Time spent starting tablet: real 0.008s user 0.005s sys 0.003s
I20260808 10:39:31.357627 3514 ts_tablet_manager.cc:1428] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260808 10:39:31.357758 3516 ts_tablet_manager.cc:1428] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260808 10:39:31.357934 3192 heartbeater.cc:503] Master 127.1.194.126:40311 was elected leader, sending a full tablet report...
I20260808 10:39:31.358067 3346 heartbeater.cc:503] Master 127.1.194.126:40311 was elected leader, sending a full tablet report...
W20260808 10:39:31.515923 3491 tablet.cc:2369] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:31.536726 3347 tablet.cc:2369] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:31.547061 3193 tablet.cc:2369] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:31.595177 3520 raft_consensus.cc:482] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:31.595430 3520 raft_consensus.cc:504] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.596671 3520 leader_election.cc:288] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025), f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291)
I20260808 10:39:31.601060 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.597039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:42025 (local address 127.1.194.67:57177)
0808 10:39:31.597224 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.597241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.597367 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.597821 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.597827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.597846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.597960 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.597964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.598765 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.598769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.599859 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.599869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.600527 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.600531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.600750 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.600768 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.600871 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.600944 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":1}
I20260808 10:39:31.601161 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.597086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:57177 (local address 127.1.194.66:42025)
0808 10:39:31.597359 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.597364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.597383 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:39:31.597598 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.597602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.597647 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.597715 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.598069 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.598609 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.600016 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.600415 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.600522 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.600561 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.600943 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.601035 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260808 10:39:31.601852 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.597096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39291 (local address 127.1.194.67:49449)
0808 10:39:31.597617 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.597629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.597772 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.598094 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.598096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.598111 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.598774 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.598782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.599585 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.599589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.600512 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.600521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.601514 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.601519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.601611 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.601628 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.601687 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.601731 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":193,"threads_started":1}
I20260808 10:39:31.601922 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.597091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:49449 (local address 127.1.194.65:39291)
0808 10:39:31.597224 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.597229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.597739 (+ 510us) server_negotiation.cc:407] Connection header received
0808 10:39:31.597904 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.597906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.597948 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.598061 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.598893 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.599465 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.600644 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.601373 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.601508 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.601558 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.601730 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.601805 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:39:31.601861 3295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325961a20bd5432b8daeffb64ac77f94" candidate_uuid: "e29e39a570ef43248ffd1eec2aa2a359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fb37abead64d859f509b346f3b7028" is_pre_election: true
I20260808 10:39:31.602226 3295 raft_consensus.cc:2454] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e29e39a570ef43248ffd1eec2aa2a359 in term 0.
I20260808 10:39:31.602821 3373 leader_election.cc:302] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fb37abead64d859f509b346f3b7028, e29e39a570ef43248ffd1eec2aa2a359; no voters:
I20260808 10:39:31.602880 3141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325961a20bd5432b8daeffb64ac77f94" candidate_uuid: "e29e39a570ef43248ffd1eec2aa2a359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" is_pre_election: true
I20260808 10:39:31.603144 3520 raft_consensus.cc:2781] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:31.603240 3520 raft_consensus.cc:482] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:31.603310 3520 raft_consensus.cc:3037] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.603255 3141 raft_consensus.cc:2454] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e29e39a570ef43248ffd1eec2aa2a359 in term 0.
I20260808 10:39:31.604651 3520 raft_consensus.cc:504] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.604966 3520 leader_election.cc:288] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [CANDIDATE]: Term 1 election: Requested vote from peers 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025), f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291)
I20260808 10:39:31.605290 3141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325961a20bd5432b8daeffb64ac77f94" candidate_uuid: "e29e39a570ef43248ffd1eec2aa2a359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1acab4297ce4a69ab5eb413a20c8a80"
I20260808 10:39:31.605278 3295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325961a20bd5432b8daeffb64ac77f94" candidate_uuid: "e29e39a570ef43248ffd1eec2aa2a359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fb37abead64d859f509b346f3b7028"
I20260808 10:39:31.605386 3141 raft_consensus.cc:3037] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.605425 3295 raft_consensus.cc:3037] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.606619 3295 raft_consensus.cc:2454] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e29e39a570ef43248ffd1eec2aa2a359 in term 1.
I20260808 10:39:31.606771 3141 raft_consensus.cc:2454] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e29e39a570ef43248ffd1eec2aa2a359 in term 1.
I20260808 10:39:31.606947 3373 leader_election.cc:302] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fb37abead64d859f509b346f3b7028, e29e39a570ef43248ffd1eec2aa2a359; no voters:
I20260808 10:39:31.607159 3520 raft_consensus.cc:2781] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:31.607452 3520 raft_consensus.cc:686] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 LEADER]: Becoming Leader. State: Replica: e29e39a570ef43248ffd1eec2aa2a359, State: Running, Role: LEADER
I20260808 10:39:31.607786 3520 consensus_queue.cc:237] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } }
I20260808 10:39:31.611016 2972 catalog_manager.cc:5495] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 reported cstate change: term changed from 0 to 1, leader changed from <none> to e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "e29e39a570ef43248ffd1eec2aa2a359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:31.631196 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.627512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.1.194.67:38839)
0808 10:39:31.627646 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.627649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.627663 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:39:31.627819 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.627821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.627875 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.627960 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:31.628251 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.628748 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.629659 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.629913 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.630025 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.630374 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.630818 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.630911 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.631032 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.631089 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:39:31.637964 3141 raft_consensus.cc:1261] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Refusing update from remote peer e29e39a570ef43248ffd1eec2aa2a359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:31.637987 3295 raft_consensus.cc:1261] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Refusing update from remote peer e29e39a570ef43248ffd1eec2aa2a359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:31.638788 3525 consensus_queue.cc:1035] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:31.640702 3525 consensus_queue.cc:1035] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:31.649376 3532 mvcc.cc:205] Tried to move back new op lower bound from 7316216101420150784 to 7316216101307838464. Current Snapshot: MvccSnapshot[applied={T|T < 7316216101420150784}]
I20260808 10:39:31.661125 2972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:31.661460 2972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:31.664537 2972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: Sending DeleteTablet for 3 replicas of tablet 325961a20bd5432b8daeffb64ac77f94
I20260808 10:39:31.665720 3423 tablet_service.cc:1514] Processing DeleteTablet for tablet 325961a20bd5432b8daeffb64ac77f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:31 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:40572
I20260808 10:39:31.665787 3119 tablet_service.cc:1514] Processing DeleteTablet for tablet 325961a20bd5432b8daeffb64ac77f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:31 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:53854
I20260808 10:39:31.666553 3272 tablet_service.cc:1514] Processing DeleteTablet for tablet 325961a20bd5432b8daeffb64ac77f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:31 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:59054
I20260808 10:39:31.666746 3540 tablet_replica.cc:330] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: stopping tablet replica
I20260808 10:39:31.667032 3542 tablet_replica.cc:330] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: stopping tablet replica
I20260808 10:39:31.666939 2976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:39:31.667343 3542 raft_consensus.cc:2229] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:39:31.667384 2976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:31.667629 3540 raft_consensus.cc:2229] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:31.667727 3542 raft_consensus.cc:2258] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:31.668248 3540 raft_consensus.cc:2258] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:31.668936 3541 tablet_replica.cc:330] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: stopping tablet replica
I20260808 10:39:31.669380 3541 raft_consensus.cc:2229] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:31.669535 3542 ts_tablet_manager.cc:1905] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:31.670177 3541 raft_consensus.cc:2258] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:31.670688 3540 ts_tablet_manager.cc:1905] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:31.673724 3542 ts_tablet_manager.cc:1918] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:31.673867 3542 log.cc:1196] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/325961a20bd5432b8daeffb64ac77f94
I20260808 10:39:31.674220 3542 ts_tablet_manager.cc:1939] T 325961a20bd5432b8daeffb64ac77f94 P 51fb37abead64d859f509b346f3b7028: Deleting consensus metadata
I20260808 10:39:31.674437 3272 tablet_service.cc:1467] Processing CreateTablet for tablet 982a0848e56545f598914ef53c318b0e (DEFAULT_TABLE table=test-table [id=949d13695ee14b16ae1ac87501475c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:31.674690 3272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982a0848e56545f598914ef53c318b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.674769 3540 ts_tablet_manager.cc:1918] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:31.674875 3540 log.cc:1196] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/325961a20bd5432b8daeffb64ac77f94
I20260808 10:39:31.675235 3540 ts_tablet_manager.cc:1939] T 325961a20bd5432b8daeffb64ac77f94 P e29e39a570ef43248ffd1eec2aa2a359: Deleting consensus metadata
I20260808 10:39:31.675980 3423 tablet_service.cc:1467] Processing CreateTablet for tablet 982a0848e56545f598914ef53c318b0e (DEFAULT_TABLE table=test-table [id=949d13695ee14b16ae1ac87501475c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:31.676245 3423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982a0848e56545f598914ef53c318b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.676620 2960 catalog_manager.cc:4841] TS 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025): tablet 325961a20bd5432b8daeffb64ac77f94 (table test-table [id=1a690423049042cb848ea11335d9fe12]) successfully deleted
I20260808 10:39:31.677248 3516 tablet_bootstrap.cc:492] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Bootstrap starting.
I20260808 10:39:31.678154 3541 ts_tablet_manager.cc:1905] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:31.678272 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 982a0848e56545f598914ef53c318b0e (DEFAULT_TABLE table=test-table [id=949d13695ee14b16ae1ac87501475c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:31.678553 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982a0848e56545f598914ef53c318b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.678755 3515 tablet_bootstrap.cc:492] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Bootstrap starting.
I20260808 10:39:31.680310 3515 tablet_bootstrap.cc:654] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.681697 3541 ts_tablet_manager.cc:1918] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:31.681661 2961 catalog_manager.cc:4841] TS e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839): tablet 325961a20bd5432b8daeffb64ac77f94 (table test-table [id=1a690423049042cb848ea11335d9fe12]) successfully deleted
I20260808 10:39:31.681820 3514 tablet_bootstrap.cc:492] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Bootstrap starting.
I20260808 10:39:31.681833 3541 log.cc:1196] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/325961a20bd5432b8daeffb64ac77f94
I20260808 10:39:31.682370 3541 ts_tablet_manager.cc:1939] T 325961a20bd5432b8daeffb64ac77f94 P f1acab4297ce4a69ab5eb413a20c8a80: Deleting consensus metadata
I20260808 10:39:31.683347 3514 tablet_bootstrap.cc:654] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.683684 3516 tablet_bootstrap.cc:654] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.684648 2963 catalog_manager.cc:4841] TS f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291): tablet 325961a20bd5432b8daeffb64ac77f94 (table test-table [id=1a690423049042cb848ea11335d9fe12]) successfully deleted
I20260808 10:39:31.685865 3515 tablet_bootstrap.cc:492] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: No bootstrap required, opened a new log
I20260808 10:39:31.685963 3515 ts_tablet_manager.cc:1397] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 10:39:31.685994 3514 tablet_bootstrap.cc:492] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: No bootstrap required, opened a new log
I20260808 10:39:31.686038 3516 tablet_bootstrap.cc:492] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: No bootstrap required, opened a new log
I20260808 10:39:31.686059 3514 ts_tablet_manager.cc:1397] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 10:39:31.686143 3516 ts_tablet_manager.cc:1397] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260808 10:39:31.686738 3514 raft_consensus.cc:348] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.686769 3515 raft_consensus.cc:348] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.686753 3516 raft_consensus.cc:348] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.686872 3515 raft_consensus.cc:374] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.686872 3514 raft_consensus.cc:374] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.686908 3516 raft_consensus.cc:374] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.686913 3515 raft_consensus.cc:729] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29e39a570ef43248ffd1eec2aa2a359, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.686932 3514 raft_consensus.cc:729] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1acab4297ce4a69ab5eb413a20c8a80, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.686942 3516 raft_consensus.cc:729] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fb37abead64d859f509b346f3b7028, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.687045 3515 consensus_queue.cc:260] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.687047 3514 consensus_queue.cc:260] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.687081 3516 consensus_queue.cc:260] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.687315 3514 ts_tablet_manager.cc:1428] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:31.687315 3515 ts_tablet_manager.cc:1428] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:39:31.687359 3516 ts_tablet_manager.cc:1428] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:39:31.735247 3523 raft_consensus.cc:482] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:31.735399 3523 raft_consensus.cc:504] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.736639 3523 leader_election.cc:288] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291), e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839)
I20260808 10:39:31.741940 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.737185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:37689 (local address 127.1.194.67:38839)
0808 10:39:31.737327 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.737333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.737981 (+ 648us) server_negotiation.cc:407] Connection header received
0808 10:39:31.738181 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.738185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.738244 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.738312 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.738895 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.739374 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.740613 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.741080 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.741214 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.741318 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.741699 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.741778 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:39:31.742419 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.736974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:38839 (local address 127.1.194.66:37689)
0808 10:39:31.737870 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.737885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.738051 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.738482 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.738484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.738498 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.738782 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.738790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.739488 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.739494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.740489 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.740499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.741305 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.741309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.741511 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.741550 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.742135 (+ 585us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.742224 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":371,"thread_start_us":235,"threads_started":1}
I20260808 10:39:31.743140 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.737102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:54899 (local address 127.1.194.65:39291)
0808 10:39:31.737263 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.737267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.737533 (+ 266us) server_negotiation.cc:407] Connection header received
0808 10:39:31.737788 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.737791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.737841 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.737887 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.738423 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.739303 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.741247 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.742280 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.742453 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.742501 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.742928 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.743006 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:39:31.743153 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.736966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39291 (local address 127.1.194.66:54899)
0808 10:39:31.737367 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.737382 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.737652 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.738076 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.738079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.738097 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.738285 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.738293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.739474 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.739477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.741085 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.741096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.742675 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.742690 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.742790 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.742807 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.742908 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.742983 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":33,"thread_start_us":84,"threads_started":1}
I20260808 10:39:31.743534 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982a0848e56545f598914ef53c318b0e" candidate_uuid: "51fb37abead64d859f509b346f3b7028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e29e39a570ef43248ffd1eec2aa2a359" is_pre_election: true
I20260808 10:39:31.743842 3441 raft_consensus.cc:2454] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51fb37abead64d859f509b346f3b7028 in term 0.
I20260808 10:39:31.743948 3141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982a0848e56545f598914ef53c318b0e" candidate_uuid: "51fb37abead64d859f509b346f3b7028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" is_pre_election: true
I20260808 10:39:31.744081 3141 raft_consensus.cc:2454] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51fb37abead64d859f509b346f3b7028 in term 0.
I20260808 10:39:31.744347 3221 leader_election.cc:302] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fb37abead64d859f509b346f3b7028, e29e39a570ef43248ffd1eec2aa2a359; no voters:
I20260808 10:39:31.744673 3523 raft_consensus.cc:2781] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:31.744726 3523 raft_consensus.cc:482] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:31.744752 3523 raft_consensus.cc:3037] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.746404 3523 raft_consensus.cc:504] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.746786 3523 leader_election.cc:288] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [CANDIDATE]: Term 1 election: Requested vote from peers f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291), e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839)
I20260808 10:39:31.747856 3141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982a0848e56545f598914ef53c318b0e" candidate_uuid: "51fb37abead64d859f509b346f3b7028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1acab4297ce4a69ab5eb413a20c8a80"
I20260808 10:39:31.747983 3141 raft_consensus.cc:3037] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.749127 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982a0848e56545f598914ef53c318b0e" candidate_uuid: "51fb37abead64d859f509b346f3b7028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e29e39a570ef43248ffd1eec2aa2a359"
I20260808 10:39:31.749289 3441 raft_consensus.cc:3037] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.749420 3141 raft_consensus.cc:2454] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51fb37abead64d859f509b346f3b7028 in term 1.
I20260808 10:39:31.749753 3223 leader_election.cc:302] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fb37abead64d859f509b346f3b7028, f1acab4297ce4a69ab5eb413a20c8a80; no voters:
I20260808 10:39:31.750782 3441 raft_consensus.cc:2454] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51fb37abead64d859f509b346f3b7028 in term 1.
I20260808 10:39:31.750980 3523 raft_consensus.cc:2781] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:31.751212 3523 raft_consensus.cc:686] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 1 LEADER]: Becoming Leader. State: Replica: 51fb37abead64d859f509b346f3b7028, State: Running, Role: LEADER
I20260808 10:39:31.751477 3523 consensus_queue.cc:237] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.754081 2976 catalog_manager.cc:5495] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51fb37abead64d859f509b346f3b7028 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "51fb37abead64d859f509b346f3b7028" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:39:31 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185570, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:31.803736 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.791376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.1.194.126:40311)
0808 10:39:31.791494 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.791498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.791780 (+ 282us) server_negotiation.cc:407] Connection header received
0808 10:39:31.792045 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.792048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.792097 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.792227 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:31.793179 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.793942 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.795316 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.795709 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.795928 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.799412 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:31.799416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:31.799419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:31.799468 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:31.801413 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.802206 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.802212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.802214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.802300 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:31.802603 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:31.802607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:31.802609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:31.802829 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:31.802899 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.803363 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.803534 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:39:31.804352 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.791231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:40311 (local address 127.0.0.1:35650)
0808 10:39:31.791661 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.791679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.791843 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.792244 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.792248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.792883 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:31.793028 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.793037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.794080 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.794084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.795131 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.795154 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.795948 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.795952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.796062 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.796847 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:31.796864 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:31.799202 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:31.801630 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.801640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.801647 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.802073 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.802406 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:31.802412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:31.802414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:31.802507 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:31.802979 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:31.802985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:31.803142 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.803990 (+ 848us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.804169 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":120,"threads_started":1}
I20260808 10:39:31.809675 2976 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:35650:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:39:31.810875 2976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:31.816516 3272 tablet_service.cc:1467] Processing CreateTablet for tablet 86efcaff33c04b668be8552c43491360 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cff4ff1cfa14b48b196298095f7f2a0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:39:31.816773 3423 tablet_service.cc:1467] Processing CreateTablet for tablet 86efcaff33c04b668be8552c43491360 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cff4ff1cfa14b48b196298095f7f2a0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:39:31.816926 3272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86efcaff33c04b668be8552c43491360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.817186 3423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86efcaff33c04b668be8552c43491360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.816478 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 86efcaff33c04b668be8552c43491360 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cff4ff1cfa14b48b196298095f7f2a0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:39:31.818779 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86efcaff33c04b668be8552c43491360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:31.819926 3516 tablet_bootstrap.cc:492] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: Bootstrap starting.
I20260808 10:39:31.820396 3515 tablet_bootstrap.cc:492] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: Bootstrap starting.
I20260808 10:39:31.820971 3516 tablet_bootstrap.cc:654] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.821758 3515 tablet_bootstrap.cc:654] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.823535 3515 tablet_bootstrap.cc:492] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: No bootstrap required, opened a new log
I20260808 10:39:31.823639 3515 ts_tablet_manager.cc:1397] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:31.824034 3516 tablet_bootstrap.cc:492] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: No bootstrap required, opened a new log
I20260808 10:39:31.824141 3516 ts_tablet_manager.cc:1397] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:39:31.824632 3516 raft_consensus.cc:348] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.824651 3515 raft_consensus.cc:348] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.824765 3516 raft_consensus.cc:374] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.824776 3515 raft_consensus.cc:374] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.824810 3516 raft_consensus.cc:729] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fb37abead64d859f509b346f3b7028, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.824816 3515 raft_consensus.cc:729] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29e39a570ef43248ffd1eec2aa2a359, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.824975 3516 consensus_queue.cc:260] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.825033 3515 consensus_queue.cc:260] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.825309 3548 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.825333 3535 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.825448 3548 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.825472 3535 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.826144 3516 ts_tablet_manager.cc:1428] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:31.826406 3514 tablet_bootstrap.cc:492] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: Bootstrap starting.
I20260808 10:39:31.827500 3514 tablet_bootstrap.cc:654] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:31.825966 3515 ts_tablet_manager.cc:1428] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:31.829854 3514 tablet_bootstrap.cc:492] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: No bootstrap required, opened a new log
I20260808 10:39:31.829958 3514 ts_tablet_manager.cc:1397] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:39:31.830461 3514 raft_consensus.cc:348] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.830598 3514 raft_consensus.cc:374] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:31.830641 3514 raft_consensus.cc:729] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1acab4297ce4a69ab5eb413a20c8a80, State: Initialized, Role: FOLLOWER
I20260808 10:39:31.830807 3514 consensus_queue.cc:260] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.831080 3521 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.831230 3521 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.832021 3514 ts_tablet_manager.cc:1428] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:31.873768 3521 raft_consensus.cc:482] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:31.873935 3521 raft_consensus.cc:504] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.875134 3521 leader_election.cc:288] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025), e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839)
I20260808 10:39:31.880859 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.875500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:35461 (local address 127.1.194.67:38839)
0808 10:39:31.875636 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.875642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.876039 (+ 397us) server_negotiation.cc:407] Connection header received
0808 10:39:31.876422 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.876425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.876464 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.876578 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.876945 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.877672 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.879521 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.880112 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.880228 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.880281 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.880646 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.880732 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:39:31.880927 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.875497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:51477 (local address 127.1.194.66:42025)
0808 10:39:31.875637 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.875642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.876084 (+ 442us) server_negotiation.cc:407] Connection header received
0808 10:39:31.876198 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.876200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.876243 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.876310 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:31.876721 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.877471 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.879439 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.880079 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.880228 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.880281 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.880732 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.880810 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:39:31.880966 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.875406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:38839 (local address 127.1.194.65:35461)
0808 10:39:31.875919 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.875937 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.876091 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.876627 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.876632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.876656 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.876833 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.876838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.877832 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.877835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.879282 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.879290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.880239 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.880244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.880474 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.880508 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.880602 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.880693 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":47,"spinlock_wait_cycles":19712,"thread_start_us":176,"threads_started":1}
I20260808 10:39:31.881434 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.875406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:42025 (local address 127.1.194.65:51477)
0808 10:39:31.875919 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.875937 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.876091 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.876350 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.876352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.876369 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:31.876539 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.876545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.877637 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.877640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.879277 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.879290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.880239 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.880244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.880469 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.880581 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.880656 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.880706 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":71,"spinlock_wait_cycles":12544,"thread_start_us":193,"threads_started":1}
I20260808 10:39:31.881454 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86efcaff33c04b668be8552c43491360" candidate_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e29e39a570ef43248ffd1eec2aa2a359" is_pre_election: true
I20260808 10:39:31.881645 3441 raft_consensus.cc:2454] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1acab4297ce4a69ab5eb413a20c8a80 in term 0.
I20260808 10:39:31.882041 3069 leader_election.cc:302] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e29e39a570ef43248ffd1eec2aa2a359, f1acab4297ce4a69ab5eb413a20c8a80; no voters:
I20260808 10:39:31.882136 3295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86efcaff33c04b668be8552c43491360" candidate_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fb37abead64d859f509b346f3b7028" is_pre_election: true
I20260808 10:39:31.882359 3521 raft_consensus.cc:2781] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:31.882364 3295 raft_consensus.cc:2454] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1acab4297ce4a69ab5eb413a20c8a80 in term 0.
I20260808 10:39:31.882439 3521 raft_consensus.cc:482] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:31.882583 3521 raft_consensus.cc:3037] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.884042 3521 raft_consensus.cc:504] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.884451 3521 leader_election.cc:288] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [CANDIDATE]: Term 1 election: Requested vote from peers 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025), e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839)
I20260808 10:39:31.884809 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86efcaff33c04b668be8552c43491360" candidate_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e29e39a570ef43248ffd1eec2aa2a359"
I20260808 10:39:31.884941 3441 raft_consensus.cc:3037] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.885774 3295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86efcaff33c04b668be8552c43491360" candidate_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fb37abead64d859f509b346f3b7028"
I20260808 10:39:31.885939 3295 raft_consensus.cc:3037] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:31.886117 3441 raft_consensus.cc:2454] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1acab4297ce4a69ab5eb413a20c8a80 in term 1.
I20260808 10:39:31.886416 3069 leader_election.cc:302] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e29e39a570ef43248ffd1eec2aa2a359, f1acab4297ce4a69ab5eb413a20c8a80; no voters:
I20260808 10:39:31.886627 3521 raft_consensus.cc:2781] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:31.886929 3521 raft_consensus.cc:686] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 LEADER]: Becoming Leader. State: Replica: f1acab4297ce4a69ab5eb413a20c8a80, State: Running, Role: LEADER
I20260808 10:39:31.887252 3521 consensus_queue.cc:237] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } }
I20260808 10:39:31.887600 3295 raft_consensus.cc:2454] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1acab4297ce4a69ab5eb413a20c8a80 in term 1.
I20260808 10:39:31.888873 3561 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: TxnStatusTablet state changed. Reason: New leader f1acab4297ce4a69ab5eb413a20c8a80. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.889014 3561 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:39:31.889447 3563 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:39:31.889712 3563 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:39:31.890163 2973 catalog_manager.cc:5495] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65). New cstate: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:31.918007 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.912633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39291 (local address 127.0.0.1:57518)
0808 10:39:31.912854 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.912873 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.912984 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.913327 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.913329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.913336 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:31.913426 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.913430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.914538 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.914542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.915711 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.915725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.916393 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.916401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.916536 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.916621 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.917666 (+ 1045us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.917683 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.917754 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.917849 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 10:39:31.918155 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.912827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.1.194.65:39291)
0808 10:39:31.912970 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.912975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.912992 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:39:31.913115 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.913118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.913203 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.913305 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:31.913583 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.914392 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.915889 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.916246 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.916348 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.916796 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.917443 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.917631 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.917810 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.917915 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:39:31.922648 3441 raft_consensus.cc:1261] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Refusing update from remote peer f1acab4297ce4a69ab5eb413a20c8a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:31.923360 3295 raft_consensus.cc:1261] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Refusing update from remote peer f1acab4297ce4a69ab5eb413a20c8a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:31.923424 3521 consensus_queue.cc:1035] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:31.924083 3521 consensus_queue.cc:1035] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:31.925511 3548 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: TxnStatusTablet state changed. Reason: New leader f1acab4297ce4a69ab5eb413a20c8a80. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.925642 3548 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.926192 3535 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: TxnStatusTablet state changed. Reason: New leader f1acab4297ce4a69ab5eb413a20c8a80. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.926321 3535 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.928445 3521 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.928555 3521 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:39:31.928927 3561 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.929013 3561 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P f1acab4297ce4a69ab5eb413a20c8a80: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:39:31.929913 3548 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.930028 3548 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P 51fb37abead64d859f509b346f3b7028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.930200 3530 mvcc.cc:205] Tried to move back new op lower bound from 7316216102588522496 to 7316216102453231616. Current Snapshot: MvccSnapshot[applied={T|T < 7316216102588522496}]
I20260808 10:39:31.931284 3528 mvcc.cc:205] Tried to move back new op lower bound from 7316216102588522496 to 7316216102453231616. Current Snapshot: MvccSnapshot[applied={T|T < 7316216102588522496}]
I20260808 10:39:31.931586 3535 tablet_replica.cc:439] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 } } peers { permanent_uuid: "51fb37abead64d859f509b346f3b7028" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 42025 } } peers { permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 } } }
I20260808 10:39:31.931691 3535 tablet_replica.cc:442] T 86efcaff33c04b668be8552c43491360 P e29e39a570ef43248ffd1eec2aa2a359: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:31.937980 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.934272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.1.194.66:42025)
0808 10:39:31.934390 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.934396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.934408 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:31.934495 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.934499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.934572 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.934682 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:31.934950 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.935453 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.936273 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.936507 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.936624 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.936914 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.937520 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.937640 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.937711 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.937767 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 10:39:31.949564 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.946035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:57503 (local address 127.1.194.65:39291)
0808 10:39:31.946185 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.946190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.946205 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:39:31.946356 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.946359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.946401 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.946479 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:31.947103 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.947623 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.948474 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.948691 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.948797 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.949078 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.949146 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.949224 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.949344 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.949392 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:39:31.949617 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.945672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39291 (local address 127.1.194.66:57503)
0808 10:39:31.946115 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.946132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.946233 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.946488 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.946491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.946499 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:31.946973 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.946981 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.947732 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.947737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.948355 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.948363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.948800 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.948804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.948895 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.948934 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.949229 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.949238 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.949321 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.949401 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":166,"threads_started":1}
I20260808 10:39:31.957777 3441 raft_consensus.cc:1261] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Refusing update from remote peer 51fb37abead64d859f509b346f3b7028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:31.958117 3141 raft_consensus.cc:1261] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Refusing update from remote peer 51fb37abead64d859f509b346f3b7028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:31.958307 3548 consensus_queue.cc:1035] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e29e39a570ef43248ffd1eec2aa2a359" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 38839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:31.958472 3523 consensus_queue.cc:1035] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1acab4297ce4a69ab5eb413a20c8a80" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:31.980643 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.977043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:56573 (local address 127.1.194.66:42025)
0808 10:39:31.977163 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:31.977170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:31.977182 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:31.977223 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:31.977226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:31.977274 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:31.977353 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:31.977655 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.978231 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.979385 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.979708 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.979844 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.980179 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.980263 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.980343 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:31.980436 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:39:31.980486 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:39:31.980679 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:31.976620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:42025 (local address 127.1.194.65:56573)
0808 10:39:31.976986 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:31.977003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:31.977113 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:31.977363 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:31.977366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:31.977373 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:31.977537 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.977544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.978345 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.978350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.979283 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:31.979294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:31.979851 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:31.979855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:31.979964 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:31.980027 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:31.980347 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:31.980356 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:39:31.980417 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:39:31.980495 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":127,"threads_started":1}
I20260808 10:39:32.008793 2976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:32.008986 2976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:32.011682 2976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78a695d3be4d45009324a708931fa2f3: Sending DeleteTablet for 3 replicas of tablet 982a0848e56545f598914ef53c318b0e
I20260808 10:39:32.012415 3119 tablet_service.cc:1514] Processing DeleteTablet for tablet 982a0848e56545f598914ef53c318b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:32 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:53854
I20260808 10:39:32.012425 3272 tablet_service.cc:1514] Processing DeleteTablet for tablet 982a0848e56545f598914ef53c318b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:32 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:59054
I20260808 10:39:32.012639 3542 tablet_replica.cc:330] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: stopping tablet replica
I20260808 10:39:32.012683 3541 tablet_replica.cc:330] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: stopping tablet replica
I20260808 10:39:32.012745 3542 raft_consensus.cc:2229] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:32.012794 3541 raft_consensus.cc:2229] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:32.012946 3541 raft_consensus.cc:2258] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:32.013036 3542 raft_consensus.cc:2258] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:32.013955 3541 ts_tablet_manager.cc:1905] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:32.014319 3542 ts_tablet_manager.cc:1905] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:32.017318 3541 ts_tablet_manager.cc:1918] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:32.017422 3541 log.cc:1196] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/982a0848e56545f598914ef53c318b0e
I20260808 10:39:32.017724 3541 ts_tablet_manager.cc:1939] T 982a0848e56545f598914ef53c318b0e P f1acab4297ce4a69ab5eb413a20c8a80: Deleting consensus metadata
I20260808 10:39:32.017747 3542 ts_tablet_manager.cc:1918] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:32.017976 3542 log.cc:1196] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/982a0848e56545f598914ef53c318b0e
I20260808 10:39:32.018572 2963 catalog_manager.cc:4841] TS f1acab4297ce4a69ab5eb413a20c8a80 (127.1.194.65:39291): tablet 982a0848e56545f598914ef53c318b0e (table test-table [id=949d13695ee14b16ae1ac87501475c78]) successfully deleted
I20260808 10:39:32.018640 3423 tablet_service.cc:1514] Processing DeleteTablet for tablet 982a0848e56545f598914ef53c318b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:32 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:40572
I20260808 10:39:32.018826 3540 tablet_replica.cc:330] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: stopping tablet replica
I20260808 10:39:32.018930 3540 raft_consensus.cc:2229] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:32.019058 3540 raft_consensus.cc:2258] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:32.019752 3542 ts_tablet_manager.cc:1939] T 982a0848e56545f598914ef53c318b0e P 51fb37abead64d859f509b346f3b7028: Deleting consensus metadata
I20260808 10:39:32.019857 3540 ts_tablet_manager.cc:1905] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:32.020798 2960 catalog_manager.cc:4841] TS 51fb37abead64d859f509b346f3b7028 (127.1.194.66:42025): tablet 982a0848e56545f598914ef53c318b0e (table test-table [id=949d13695ee14b16ae1ac87501475c78]) successfully deleted
I20260808 10:39:32.023036 3540 ts_tablet_manager.cc:1918] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:32.023145 3540 log.cc:1196] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/982a0848e56545f598914ef53c318b0e
I20260808 10:39:32.023425 3540 ts_tablet_manager.cc:1939] T 982a0848e56545f598914ef53c318b0e P e29e39a570ef43248ffd1eec2aa2a359: Deleting consensus metadata
I20260808 10:39:32.024302 2961 catalog_manager.cc:4841] TS e29e39a570ef43248ffd1eec2aa2a359 (127.1.194.67:38839): tablet 982a0848e56545f598914ef53c318b0e (table test-table [id=949d13695ee14b16ae1ac87501475c78]) successfully deleted
Aug 08 10:39:32 dist-test-slave-ss4n krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185571, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:32.039311 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:32.024561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.1.194.65:39291)
0808 10:39:32.024788 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:32.024794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:32.025262 (+ 468us) server_negotiation.cc:407] Connection header received
0808 10:39:32.025436 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:32.025439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:32.025524 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:32.025613 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:32.026643 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:32.027431 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:32.028966 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:32.029322 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:32.029444 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:32.033915 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:32.033951 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:32.033972 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:32.034013 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:32.036466 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:32.037045 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:32.037051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:32.037053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:32.037226 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:32.037836 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:32.037840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:32.037844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:32.038332 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:32.038533 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:32.038974 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 10:39:32.039132 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260808 10:39:32.040934 3185 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57524
I20260808 10:39:32.053674 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:32.043516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.1.194.126:40311)
0808 10:39:32.043653 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:32.043658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:32.044073 (+ 415us) server_negotiation.cc:407] Connection header received
0808 10:39:32.044288 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:32.044291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:32.044346 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:32.044453 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:32.045380 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:32.046240 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:32.047432 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:32.047772 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:32.047848 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:32.049814 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:32.049818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:32.049821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:32.049849 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:32.051645 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:32.052169 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:32.052173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:32.052175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:32.052260 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:32.052551 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:32.052555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:32.052556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:32.052775 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:32.052939 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:32.053341 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 10:39:32.053469 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:39:32.054529 2976 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40850
I20260808 10:39:32.056212 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 3052
I20260808 10:39:32.069346 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 3201
I20260808 10:39:32.077768 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 3356
I20260808 10:39:32.086521 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 2942
2026-08-08T10:39:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2979 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:32 dist-test-slave-ss4n krb5kdc[3586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:32 dist-test-slave-ss4n krb5kdc[3586](info): set up 2 sockets
Aug 08 10:39:32 dist-test-slave-ss4n krb5kdc[3586](info): commencing operation
krb5kdc: starting...
W20260808 10:39:33.226243 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:33 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:33.361136 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46195
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:46195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:33.474287 3599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:33.474627 3599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:33.474705 3599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:33.478884 3599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:33.478982 3599 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:39:33.479013 3599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:33.479053 3599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:33.479089 3599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:33.479125 3599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:33.484462 3599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:46195
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:33.485589 3599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:33.486666 3599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:33.495520 3604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:33.495536 3605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:33.495520 3607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:33.495978 3599 server_base.cc:1034] running on GCE node
I20260808 10:39:33.496513 3599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:33.497733 3599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:33.498894 3599 hybrid_clock.cc:648] HybridClock initialized: now 1786185573498867 us; error 40 us; skew 500 ppm
Aug 08 10:39:33 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:33.506724 3599 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:33.507704 3599 webserver.cc:460] Webserver started at http://127.1.194.126:44807/ using document root <none> and password file <none>
I20260808 10:39:33.508260 3599 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:33.508318 3599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:33.508496 3599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:33.510588 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "3c2ef721ae78489ab9ac40ecba38657b"
format_stamp: "Formatted at 2026-08-08 10:39:33 on dist-test-slave-ss4n"
server_key: "8be88efef2913ab4ad7dd2190def72ce"
server_key_iv: "c3a699ae6e38fcfd41f0ad26cf16e829"
server_key_version: "encryptionkey@0"
I20260808 10:39:33.511101 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "3c2ef721ae78489ab9ac40ecba38657b"
format_stamp: "Formatted at 2026-08-08 10:39:33 on dist-test-slave-ss4n"
server_key: "8be88efef2913ab4ad7dd2190def72ce"
server_key_iv: "c3a699ae6e38fcfd41f0ad26cf16e829"
server_key_version: "encryptionkey@0"
I20260808 10:39:33.514829 3599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:33.517776 3615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:33.519173 3599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:33.519335 3599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "3c2ef721ae78489ab9ac40ecba38657b"
format_stamp: "Formatted at 2026-08-08 10:39:33 on dist-test-slave-ss4n"
server_key: "8be88efef2913ab4ad7dd2190def72ce"
server_key_iv: "c3a699ae6e38fcfd41f0ad26cf16e829"
server_key_version: "encryptionkey@0"
I20260808 10:39:33.519474 3599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:33.535207 3599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:33.538704 3599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:33.538926 3599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:33.547449 3599 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:46195
I20260808 10:39:33.547467 3680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:46195 every 8 connection(s)
I20260808 10:39:33.548641 3599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:33.548913 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 3599
I20260808 10:39:33.549012 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:33.549381 1801 external_mini_cluster.cc:1428] Setting key a1c2a4d4d8bb109e8757f83327c558e4
I20260808 10:39:33.552071 3681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:33.558499 3681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Bootstrap starting.
Aug 08 10:39:33 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:33.561065 3681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:33.562614 3681 log.cc:824] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:33.565214 3681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: No bootstrap required, opened a new log
I20260808 10:39:33.566597 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:33.550924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.1.194.126:46195)
0808 10:39:33.551494 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:33.551516 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:33.551598 (+ 82us) server_negotiation.cc:407] Connection header received
0808 10:39:33.552628 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:33.552657 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:33.553012 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:33.553323 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:33.554275 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.555497 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.556714 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.557231 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.557367 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:33.560446 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:33.560452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:33.560467 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:33.560498 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:33.563026 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:33.563700 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:33.563706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:33.563712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:33.563867 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:33.564308 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:33.564311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:33.564313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:33.564827 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:33.565029 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:33.565326 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 10:39:33.565674 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":151,"threads_started":1}
I20260808 10:39:33.568924 3681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2ef721ae78489ab9ac40ecba38657b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46195 } }
I20260808 10:39:33.569167 3681 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:33.569207 3681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2ef721ae78489ab9ac40ecba38657b, State: Initialized, Role: FOLLOWER
I20260808 10:39:33.569607 3681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2ef721ae78489ab9ac40ecba38657b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46195 } }
I20260808 10:39:33.569728 3681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:33.569768 3681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:33.569837 3681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:33.570703 3681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2ef721ae78489ab9ac40ecba38657b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46195 } }
I20260808 10:39:33.570987 3681 leader_election.cc:302] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c2ef721ae78489ab9ac40ecba38657b; no voters:
I20260808 10:39:33.571285 3681 leader_election.cc:288] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:33.571394 3687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:33.571624 3687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [term 1 LEADER]: Becoming Leader. State: Replica: 3c2ef721ae78489ab9ac40ecba38657b, State: Running, Role: LEADER
I20260808 10:39:33.571908 3687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2ef721ae78489ab9ac40ecba38657b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46195 } }
I20260808 10:39:33.572638 3681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:33.573128 3689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c2ef721ae78489ab9ac40ecba38657b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2ef721ae78489ab9ac40ecba38657b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46195 } } }
I20260808 10:39:33.573246 3689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:33.573604 3688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c2ef721ae78489ab9ac40ecba38657b. Latest consensus state: current_term: 1 leader_uuid: "3c2ef721ae78489ab9ac40ecba38657b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2ef721ae78489ab9ac40ecba38657b" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46195 } } }
I20260808 10:39:33.573685 3691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:33.573714 3688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:33.576967 3691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:33.582870 3691 catalog_manager.cc:1347] Generated new cluster ID: 3aed0ba7496a432299305703d61defa0
I20260808 10:39:33.582943 3691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:33.589932 3691 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:33.591009 3691 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:33.597262 3691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Generated new TSK 0
I20260808 10:39:33.598217 3691 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:33.632925 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:46195
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:33.752225 3709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:33.752548 3709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:33.752653 3709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:33.757987 3709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:33.758082 3709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:33.758157 3709 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:39:33.758205 3709 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:39:33.758270 3709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:33.763765 3709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:46195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:33.764883 3709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:33.765995 3709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:33.776011 3715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:33.776079 3714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:33.776175 3709 server_base.cc:1034] running on GCE node
W20260808 10:39:33.776103 3717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:33.776759 3709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:33.777446 3709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:33.778666 3709 hybrid_clock.cc:648] HybridClock initialized: now 1786185573778648 us; error 33 us; skew 500 ppm
Aug 08 10:39:33 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:33.781853 3709 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:33.782974 3709 webserver.cc:460] Webserver started at http://127.1.194.65:35113/ using document root <none> and password file <none>
I20260808 10:39:33.783607 3709 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:33.783684 3709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:33.783936 3709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:33.786255 3709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "add1dfb4998b43d784117053a50d3cd8"
format_stamp: "Formatted at 2026-08-08 10:39:33 on dist-test-slave-ss4n"
server_key: "515f434da9beffbbec2446c133692860"
server_key_iv: "2593e8d14ab18d3b541f39e4f9f8726f"
server_key_version: "encryptionkey@0"
I20260808 10:39:33.786931 3709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "add1dfb4998b43d784117053a50d3cd8"
format_stamp: "Formatted at 2026-08-08 10:39:33 on dist-test-slave-ss4n"
server_key: "515f434da9beffbbec2446c133692860"
server_key_iv: "2593e8d14ab18d3b541f39e4f9f8726f"
server_key_version: "encryptionkey@0"
I20260808 10:39:33.791266 3709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:33.793993 3724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:33.795346 3709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:33.795506 3709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "add1dfb4998b43d784117053a50d3cd8"
format_stamp: "Formatted at 2026-08-08 10:39:33 on dist-test-slave-ss4n"
server_key: "515f434da9beffbbec2446c133692860"
server_key_iv: "2593e8d14ab18d3b541f39e4f9f8726f"
server_key_version: "encryptionkey@0"
I20260808 10:39:33.795637 3709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:33.818819 3709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:33.822319 3709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:33.822561 3709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:33.824051 3709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:33.824124 3709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:33.824196 3709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:33.824239 3709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:39:33 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:33.835940 3709 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:34707
I20260808 10:39:33.836050 3849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:34707 every 8 connection(s)
I20260808 10:39:33.837081 3709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:33.839205 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:33.825037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:50717 (local address 127.1.194.126:46195)
0808 10:39:33.825198 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:33.825201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:33.826239 (+ 1038us) server_negotiation.cc:407] Connection header received
0808 10:39:33.827315 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:33.827319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:33.827398 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:33.827519 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:33.828958 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.829766 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.831530 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.831800 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.831940 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:33.835127 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:33.835131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:33.835134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:33.835164 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:33.836938 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:33.837551 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:33.837555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:33.837557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:33.837649 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:33.838085 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:33.838088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:33.838090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:33.838287 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:33.838400 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:33.838885 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 10:39:33.839021 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:39:33.839946 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:33.825554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.1.194.65:50717)
0808 10:39:33.826075 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:33.826119 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:33.827020 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:33.827862 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:33.827869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:33.828206 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:33.828758 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:33.828775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.829980 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.829984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:33.831389 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:33.831400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.832000 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.832005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:33.832195 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:33.832855 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:33.832875 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:33.834939 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:33.837077 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:33.837081 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:33.837092 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:33.837401 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:33.837774 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:33.837778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:33.837783 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:33.837952 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:33.838399 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:33.838406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:33.838647 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:39:33.838826 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 10:39:33.839153 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":138,"threads_started":1}
I20260808 10:39:33.841048 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 3709
I20260808 10:39:33.841199 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:33.841569 1801 external_mini_cluster.cc:1428] Setting key 7b7569678394d591c60e6ceb1943024a
I20260808 10:39:33.841568 3850 heartbeater.cc:347] Connected to a master server at 127.1.194.126:46195
I20260808 10:39:33.841720 3850 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:33.842344 3850 heartbeater.cc:511] Master 127.1.194.126:46195 requested a full tablet report, sending...
I20260808 10:39:33.844059 3632 ts_manager.cc:194] Registered new tserver with Master: add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707)
I20260808 10:39:33.845456 3632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:50717
I20260808 10:39:33.852708 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:33.843991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.1.194.65:51395)
0808 10:39:33.844313 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:33.844331 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:33.844442 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:33.844703 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:33.844709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:33.844985 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:33.845157 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:33.845169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.846296 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.846302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:33.847168 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:33.847178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.847777 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.847780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:33.847849 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:33.848404 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:33.848415 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:33.848814 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:33.851157 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:33.851160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:33.851161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:33.851353 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:33.851684 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:33.851686 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:33.851687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:33.851759 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:33.852253 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:33.852255 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:33.852327 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 10:39:33.852409 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:39:33.852511 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
I20260808 10:39:33.852825 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:33.843785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:51395 (local address 127.1.194.126:46195)
0808 10:39:33.843899 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:33.843904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:33.844426 (+ 522us) server_negotiation.cc:407] Connection header received
0808 10:39:33.844533 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:33.844536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:33.844580 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:33.844668 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:33.845294 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.846143 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.847299 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:33.847649 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:33.847793 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:33.848964 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:33.848971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:33.848978 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:33.849013 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:33.851028 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:33.851458 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:33.851465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:33.851469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:33.851575 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:33.851855 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:33.851862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:33.851867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:33.852146 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:33.852267 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:33.852509 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 10:39:33.852646 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:33.877249 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:46195
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:33.994740 3859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:33.994992 3859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:33.995067 3859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:33.999145 3859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:33.999203 3859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:33.999284 3859 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:39:33.999322 3859 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:39:33.999356 3859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:34.004545 3859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:46195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:34.005550 3859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:34.006533 3859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:34.016394 3864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:34.016374 3867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:34.016374 3865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:34.017002 3859 server_base.cc:1034] running on GCE node
I20260808 10:39:34.017411 3859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:34.018072 3859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:34.019302 3859 hybrid_clock.cc:648] HybridClock initialized: now 1786185574019205 us; error 114 us; skew 500 ppm
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:34.022490 3859 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:34.023414 3859 webserver.cc:460] Webserver started at http://127.1.194.66:36565/ using document root <none> and password file <none>
I20260808 10:39:34.024099 3859 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:34.024163 3859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:34.024384 3859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:34.026607 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "a23777707f4240a29bcf5f0e2f25eba0"
format_stamp: "Formatted at 2026-08-08 10:39:34 on dist-test-slave-ss4n"
server_key: "1331a0276c5e90af3365b1ba9a835e9a"
server_key_iv: "b324de078d3fa2cbe3914b365dfe40d6"
server_key_version: "encryptionkey@0"
I20260808 10:39:34.027160 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "a23777707f4240a29bcf5f0e2f25eba0"
format_stamp: "Formatted at 2026-08-08 10:39:34 on dist-test-slave-ss4n"
server_key: "1331a0276c5e90af3365b1ba9a835e9a"
server_key_iv: "b324de078d3fa2cbe3914b365dfe40d6"
server_key_version: "encryptionkey@0"
I20260808 10:39:34.030922 3859 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:39:34.033365 3875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:34.034847 3859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:34.034961 3859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "a23777707f4240a29bcf5f0e2f25eba0"
format_stamp: "Formatted at 2026-08-08 10:39:34 on dist-test-slave-ss4n"
server_key: "1331a0276c5e90af3365b1ba9a835e9a"
server_key_iv: "b324de078d3fa2cbe3914b365dfe40d6"
server_key_version: "encryptionkey@0"
I20260808 10:39:34.035106 3859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:34.058728 3859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:34.062196 3859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:34.062413 3859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:34.064055 3859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:34.064133 3859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:34.064204 3859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:34.064245 3859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:34.075685 3859 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:38509
I20260808 10:39:34.075764 3995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:38509 every 8 connection(s)
I20260808 10:39:34.076838 3859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:34.078877 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.064970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:60759 (local address 127.1.194.126:46195)
0808 10:39:34.065119 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.065123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.066279 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 10:39:34.067467 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.067473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.067544 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.067647 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:34.069158 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.070116 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.071513 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.071756 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.071872 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.074806 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:34.074810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:34.074812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:34.074841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:34.076684 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.077335 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.077338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.077340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.077430 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.077835 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.077838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.077840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.078054 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:34.078141 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.078652 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.078762 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:39:34.079685 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.065559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.1.194.66:60759)
0808 10:39:34.066136 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.066179 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.067212 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.067882 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.067891 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.068316 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:34.068979 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.068999 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.070280 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.070283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.071384 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.071394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.071923 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.071928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.072108 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.072723 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:34.072738 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:34.074600 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:34.076808 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.076813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.076823 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.077202 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.077572 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.077575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.077577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.077740 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.078149 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:34.078154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:34.078417 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.078613 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.078856 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":139,"threads_started":1}
I20260808 10:39:34.080989 3996 heartbeater.cc:347] Connected to a master server at 127.1.194.126:46195
I20260808 10:39:34.081158 3996 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:34.081786 3996 heartbeater.cc:511] Master 127.1.194.126:46195 requested a full tablet report, sending...
I20260808 10:39:34.082827 3633 ts_manager.cc:194] Registered new tserver with Master: a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509)
I20260808 10:39:34.083587 3633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:60759
I20260808 10:39:34.084791 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 3859
I20260808 10:39:34.084892 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:34.085223 1801 external_mini_cluster.cc:1428] Setting key 391b8a0d4674ba85194f9b90b0a974b0
I20260808 10:39:34.091400 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.083087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.1.194.66:39379)
0808 10:39:34.083428 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.083440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.083519 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.083785 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.083787 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.084029 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:34.084155 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.084161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.085094 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.085099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.085940 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.085950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.086401 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.086404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.086495 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.087064 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:34.087084 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:34.087537 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:34.089921 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.089923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.089925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.090120 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.090420 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.090422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.090423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.090495 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.090997 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:34.090999 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:34.091079 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.091163 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.091265 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":93,"threads_started":1}
I20260808 10:39:34.091615 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.083166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:39379 (local address 127.1.194.126:46195)
0808 10:39:34.083297 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.083301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.083514 (+ 213us) server_negotiation.cc:407] Connection header received
0808 10:39:34.083586 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.083589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.083640 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.083742 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:34.084268 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.084924 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.086072 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.086289 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.086397 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.087715 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:34.087718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:34.087720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:34.087749 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:34.089812 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.090222 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.090226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.090227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.090323 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.090580 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.090586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.090590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.090894 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:34.091005 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.091245 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.091374 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:34.117329 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:46195
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:34.228549 4006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:34.228812 4006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:34.228861 4006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:34.232906 4006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:34.232962 4006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:34.233004 4006 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:39:34.233018 4006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:39:34.233108 4006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:34.238296 4006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:46195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:34.239399 4006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:34.240394 4006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:34.250572 4013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:34.250566 4012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:34.250566 4015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:34.250854 4006 server_base.cc:1034] running on GCE node
I20260808 10:39:34.251248 4006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:34.251894 4006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:34.253083 4006 hybrid_clock.cc:648] HybridClock initialized: now 1786185574253058 us; error 60 us; skew 500 ppm
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:34.256000 4006 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:34.256830 4006 webserver.cc:460] Webserver started at http://127.1.194.67:42021/ using document root <none> and password file <none>
I20260808 10:39:34.257369 4006 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:34.257459 4006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:34.257733 4006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:34.259706 4006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
format_stamp: "Formatted at 2026-08-08 10:39:34 on dist-test-slave-ss4n"
server_key: "37afa0e977cf0867584e87e45a8c46a9"
server_key_iv: "0fce1a1700ddc594dde476259c801e8e"
server_key_version: "encryptionkey@0"
I20260808 10:39:34.260368 4006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
format_stamp: "Formatted at 2026-08-08 10:39:34 on dist-test-slave-ss4n"
server_key: "37afa0e977cf0867584e87e45a8c46a9"
server_key_iv: "0fce1a1700ddc594dde476259c801e8e"
server_key_version: "encryptionkey@0"
I20260808 10:39:34.264227 4006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 10:39:34.266844 4022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:34.268103 4006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:34.268318 4006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
format_stamp: "Formatted at 2026-08-08 10:39:34 on dist-test-slave-ss4n"
server_key: "37afa0e977cf0867584e87e45a8c46a9"
server_key_iv: "0fce1a1700ddc594dde476259c801e8e"
server_key_version: "encryptionkey@0"
I20260808 10:39:34.268435 4006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:34.282155 4006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:34.285539 4006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:34.285749 4006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:34.287292 4006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:34.287369 4006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:34.287436 4006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:34.287480 4006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:34.298956 4006 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:42507
I20260808 10:39:34.299999 4006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:34.300560 4153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:42507 every 8 connection(s)
I20260808 10:39:34.301913 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.288146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:53999 (local address 127.1.194.126:46195)
0808 10:39:34.288283 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.288286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.289346 (+ 1060us) server_negotiation.cc:407] Connection header received
0808 10:39:34.290573 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.290578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.290649 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.290698 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:34.292212 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.293004 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.294383 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.294712 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.294834 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.297786 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:34.297789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:34.297792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:34.297826 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:34.299588 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.300221 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.300228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.300230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.300324 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.300793 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.300796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.300798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.301021 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:34.301130 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.301692 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.301795 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:39:34.302701 4031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.288654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.1.194.67:53999)
0808 10:39:34.289151 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.289235 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.290299 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.291065 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.291071 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.291447 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:34.292072 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.292086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.293133 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.293136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.294247 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.294261 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.294882 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.294886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.295055 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.295670 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:34.295690 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:34.297627 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:34.299740 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.299745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.299758 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.300098 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.300447 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.300452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.300455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.300662 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.301142 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:34.301150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:34.301385 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.301596 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.301835 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":143,"threads_started":1}
I20260808 10:39:34.303902 4154 heartbeater.cc:347] Connected to a master server at 127.1.194.126:46195
I20260808 10:39:34.304057 4154 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:34.304577 4154 heartbeater.cc:511] Master 127.1.194.126:46195 requested a full tablet report, sending...
I20260808 10:39:34.304616 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 4006
I20260808 10:39:34.304761 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:34.305137 1801 external_mini_cluster.cc:1428] Setting key 1d858ac35de5224d7264adce70a66c83
I20260808 10:39:34.305716 3632 ts_manager.cc:194] Registered new tserver with Master: 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.306480 3632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:53999
I20260808 10:39:34.307199 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:39:34.314795 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.306277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:59593 (local address 127.1.194.126:46195)
0808 10:39:34.306392 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.306396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.306648 (+ 252us) server_negotiation.cc:407] Connection header received
0808 10:39:34.306810 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.306813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.306860 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.306943 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:34.307509 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.308260 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.309610 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.309969 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.310084 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.311237 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:34.311243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:34.311250 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:34.311280 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:34.313108 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.313628 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.313634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.313639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.313722 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.314038 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.314041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.314042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.314200 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:34.314276 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.314592 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.314686 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:34.314990 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.306222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.1.194.67:59593)
0808 10:39:34.306550 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.306584 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.306715 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.306959 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.306961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.307225 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:34.307389 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.307397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.308407 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.308414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.309412 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.309426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.310099 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.310102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.310193 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.310741 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:34.310755 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:34.311096 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:34.313224 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.313226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.313233 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.313423 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.313831 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.313835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.313837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.313933 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.314298 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:34.314304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:34.314423 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.314544 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.314711 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:34.336212 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.327787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.1.194.126:46195)
0808 10:39:34.327899 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.327902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.327916 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:39:34.327956 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.327958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.327996 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.328068 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:34.328795 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.329316 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.330203 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.330413 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.330494 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.332881 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:34.332884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:34.332887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:34.332910 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:34.334259 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.334785 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.334789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.334790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.334899 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.335203 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.335206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.335207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.335420 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:34.335610 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.335857 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.335968 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:39:34.339344 3633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:34.342746 3633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:34.357364 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.351828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.1.194.67:42507)
0808 10:39:34.352457 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.352470 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.352492 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:39:34.352584 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.352589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.352773 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.353036 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.353321 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.354531 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356072 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.356808 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356909 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.357052 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.357147 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.357236 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":153,"threads_started":1}
I20260808 10:39:34.357350 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.351360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:42507 (local address 127.0.0.1:56754)
0808 10:39:34.352080 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.352095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.352180 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.352934 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.352937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.352964 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.353192 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.353200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.354681 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.354684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.355930 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.355941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.356911 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.357000 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.357021 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.357118 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.357193 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":133,"spinlock_wait_cycles":7936,"thread_start_us":95,"threads_started":1}
I20260808 10:39:34.357527 4179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.351902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36194 (local address 127.1.194.65:34707)
0808 10:39:34.352268 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.352275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.352510 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:39:34.352585 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.352589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.352740 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.352892 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.353457 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.354223 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.355761 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.356555 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356670 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.356820 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.357272 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.357375 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":75,"threads_started":1}
I20260808 10:39:34.357630 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.351233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.1.194.66:38509)
0808 10:39:34.352529 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.352539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.352556 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:39:34.352620 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.352624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.352773 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.353187 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.353338 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.354541 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356140 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.356731 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.357167 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.357327 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.357369 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.357447 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":152,"threads_started":1}
I20260808 10:39:34.357988 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.351756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:34707 (local address 127.0.0.1:36194)
0808 10:39:34.352303 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.352317 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.352415 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.352878 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.352884 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.352906 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.353352 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.353357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.354346 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.354350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.355628 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.355639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.356686 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.356823 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.356875 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.356977 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.357837 (+ 860us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":279,"thread_start_us":103,"threads_started":1}
I20260808 10:39:34.357980 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.351033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38509 (local address 127.0.0.1:48456)
0808 10:39:34.351380 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.351436 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.351560 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.352940 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.352943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.352969 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.353192 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.353200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.354681 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.354684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.356001 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.356011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.356864 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.356867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.356957 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.356983 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.357023 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.357145 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":219,"spinlock_wait_cycles":8704,"thread_start_us":83,"threads_started":1}
I20260808 10:39:34.360656 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 05972afcb58046f88990d1fc904a6082 (DEFAULT_TABLE table=test-table [id=8addf33fc9ca427880b3af2b842ed559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:34.360808 4078 tablet_service.cc:1467] Processing CreateTablet for tablet 05972afcb58046f88990d1fc904a6082 (DEFAULT_TABLE table=test-table [id=8addf33fc9ca427880b3af2b842ed559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:34.360808 3928 tablet_service.cc:1467] Processing CreateTablet for tablet 05972afcb58046f88990d1fc904a6082 (DEFAULT_TABLE table=test-table [id=8addf33fc9ca427880b3af2b842ed559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:34.361636 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05972afcb58046f88990d1fc904a6082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.361790 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05972afcb58046f88990d1fc904a6082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.361786 3928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05972afcb58046f88990d1fc904a6082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.368453 4180 tablet_bootstrap.cc:492] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Bootstrap starting.
I20260808 10:39:34.370075 4183 tablet_bootstrap.cc:492] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Bootstrap starting.
I20260808 10:39:34.370078 4182 tablet_bootstrap.cc:492] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Bootstrap starting.
I20260808 10:39:34.370684 4180 tablet_bootstrap.cc:654] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.371410 4180 log.cc:824] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:34.372102 4183 tablet_bootstrap.cc:654] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.372102 4182 tablet_bootstrap.cc:654] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.372917 4183 log.cc:824] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:34.372968 4182 log.cc:824] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:34.373974 4180 tablet_bootstrap.cc:492] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: No bootstrap required, opened a new log
I20260808 10:39:34.374305 4180 ts_tablet_manager.cc:1397] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:39:34.374954 4183 tablet_bootstrap.cc:492] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: No bootstrap required, opened a new log
I20260808 10:39:34.374993 4182 tablet_bootstrap.cc:492] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: No bootstrap required, opened a new log
I20260808 10:39:34.375134 4183 ts_tablet_manager.cc:1397] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:39:34.375134 4182 ts_tablet_manager.cc:1397] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:39:34.377871 4180 raft_consensus.cc:348] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.378123 4180 raft_consensus.cc:374] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.378183 4180 raft_consensus.cc:729] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.378593 4180 consensus_queue.cc:260] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.378957 4182 raft_consensus.cc:348] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.378957 4183 raft_consensus.cc:348] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.379194 4182 raft_consensus.cc:374] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.379194 4183 raft_consensus.cc:374] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.379244 4182 raft_consensus.cc:729] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1cbf39e53d44fa9aa713b09fd4ebf8, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.379257 4183 raft_consensus.cc:729] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.379612 4180 ts_tablet_manager.cc:1428] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:39:34.379642 4183 consensus_queue.cc:260] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.379642 4182 consensus_queue.cc:260] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.380122 3850 heartbeater.cc:503] Master 127.1.194.126:46195 was elected leader, sending a full tablet report...
I20260808 10:39:34.380544 4183 ts_tablet_manager.cc:1428] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 10:39:34.380558 4182 ts_tablet_manager.cc:1428] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260808 10:39:34.380874 3996 heartbeater.cc:503] Master 127.1.194.126:46195 was elected leader, sending a full tablet report...
I20260808 10:39:34.381940 4154 heartbeater.cc:503] Master 127.1.194.126:46195 was elected leader, sending a full tablet report...
I20260808 10:39:34.477414 4188 raft_consensus.cc:482] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:34.477710 4188 raft_consensus.cc:504] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.479008 4188 leader_election.cc:288] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.483865 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.479343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:34707 (local address 127.1.194.66:55987)
0808 10:39:34.479541 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.479559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.479659 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.479950 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.479953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.480008 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.480225 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.480232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.481517 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.481521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.482530 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.482539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.483438 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.483442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.483544 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.483570 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.483649 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.483717 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"spinlock_wait_cycles":4992}
I20260808 10:39:34.483902 4179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.479499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:55987 (local address 127.1.194.65:34707)
0808 10:39:34.479627 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.479632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.479647 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:39:34.479758 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.479762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.479814 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.479893 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.480417 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.481376 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.482678 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.483315 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.483416 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.483466 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.483733 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.483804 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:39:34.484540 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.479468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:35153 (local address 127.1.194.67:42507)
0808 10:39:34.479652 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.479658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.480024 (+ 366us) server_negotiation.cc:407] Connection header received
0808 10:39:34.480236 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.480240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.480297 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.480378 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.481247 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.482009 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.483311 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.483932 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.484052 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.484108 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.484394 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.484455 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 10:39:34.484570 4191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.479345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:42507 (local address 127.1.194.66:35153)
0808 10:39:34.479909 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.479921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.480116 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.480419 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.480422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.480444 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.481107 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.481118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.482150 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.482152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.483179 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.483189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.484177 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.484180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.484249 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.484265 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.484359 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.484423 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":1,"spinlock_wait_cycles":116608,"thread_start_us":216,"threads_started":1}
I20260808 10:39:34.484633 3800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05972afcb58046f88990d1fc904a6082" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add1dfb4998b43d784117053a50d3cd8" is_pre_election: true
I20260808 10:39:34.484984 3800 raft_consensus.cc:2454] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 0.
I20260808 10:39:34.485286 4100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05972afcb58046f88990d1fc904a6082" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" is_pre_election: true
I20260808 10:39:34.485529 3878 leader_election.cc:302] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23777707f4240a29bcf5f0e2f25eba0, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:34.485595 4100 raft_consensus.cc:2454] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 0.
I20260808 10:39:34.485802 4188 raft_consensus.cc:2781] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:34.485903 4188 raft_consensus.cc:482] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:34.485961 4188 raft_consensus.cc:3037] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.487008 4188 raft_consensus.cc:504] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.487396 4188 leader_election.cc:288] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 election: Requested vote from peers add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.487792 3800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05972afcb58046f88990d1fc904a6082" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add1dfb4998b43d784117053a50d3cd8"
I20260808 10:39:34.487852 4100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05972afcb58046f88990d1fc904a6082" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
I20260808 10:39:34.487951 3800 raft_consensus.cc:3037] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.487998 4100 raft_consensus.cc:3037] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.489004 3800 raft_consensus.cc:2454] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 1.
I20260808 10:39:34.489074 4100 raft_consensus.cc:2454] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 1.
I20260808 10:39:34.489311 3878 leader_election.cc:302] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23777707f4240a29bcf5f0e2f25eba0, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:34.489511 4188 raft_consensus.cc:2781] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:34.489749 4188 raft_consensus.cc:686] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 LEADER]: Becoming Leader. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Running, Role: LEADER
I20260808 10:39:34.490080 4188 consensus_queue.cc:237] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.493366 3632 catalog_manager.cc:5495] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:39:34 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:34.535686 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.527389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.1.194.126:46195)
0808 10:39:34.527573 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.527578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.527698 (+ 120us) server_negotiation.cc:407] Connection header received
0808 10:39:34.527875 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.527878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.527913 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.527995 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:34.528664 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.529205 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.530031 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.530241 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.530346 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.532601 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:34.532605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:34.532614 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:34.532634 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:34.534172 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.534597 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.534599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.534601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.534677 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:34.534911 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:34.534913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:34.534914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:34.535093 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:34.535173 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.535442 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.535559 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":50}
I20260808 10:39:34.535691 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.527312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46195 (local address 127.0.0.1:40336)
0808 10:39:34.527589 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.527604 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.527754 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.528014 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.528025 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.528444 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:34.528541 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.528548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.529323 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.529329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.529912 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.529920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.530349 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.530353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.530457 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.531063 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:34.531076 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:34.532462 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:34.534266 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.534273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.534278 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.534496 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.534757 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:34.534759 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:34.534760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:34.534836 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:34.535189 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:34.535194 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:34.535294 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.535385 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.535505 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260808 10:39:34.539836 3632 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:40336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:39:34.540604 3632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:34.546029 3928 tablet_service.cc:1467] Processing CreateTablet for tablet c1b969f5cdaa4a99b430e4447a7fdd70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddd51c6c6892432c9016b9d7dd972693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:39:34.545993 3779 tablet_service.cc:1467] Processing CreateTablet for tablet c1b969f5cdaa4a99b430e4447a7fdd70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddd51c6c6892432c9016b9d7dd972693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:39:34.546312 4078 tablet_service.cc:1467] Processing CreateTablet for tablet c1b969f5cdaa4a99b430e4447a7fdd70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddd51c6c6892432c9016b9d7dd972693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:39:34.546824 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b969f5cdaa4a99b430e4447a7fdd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.546829 3928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b969f5cdaa4a99b430e4447a7fdd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.546865 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b969f5cdaa4a99b430e4447a7fdd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.549839 4182 tablet_bootstrap.cc:492] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Bootstrap starting.
I20260808 10:39:34.549889 4183 tablet_bootstrap.cc:492] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: Bootstrap starting.
I20260808 10:39:34.550740 4183 tablet_bootstrap.cc:654] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.550810 4182 tablet_bootstrap.cc:654] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Neither blocks nor log segments found. Creating new log.
W20260808 10:39:34.551213 4155 tablet.cc:2369] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:34.551891 4183 tablet_bootstrap.cc:492] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: No bootstrap required, opened a new log
I20260808 10:39:34.552003 4183 ts_tablet_manager.cc:1397] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:39:34.552287 4180 tablet_bootstrap.cc:492] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: Bootstrap starting.
I20260808 10:39:34.552489 4183 raft_consensus.cc:348] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.552608 4183 raft_consensus.cc:374] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.552639 4183 raft_consensus.cc:729] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.552774 4183 consensus_queue.cc:260] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.553040 4183 ts_tablet_manager.cc:1428] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:39:34.553043 4182 tablet_bootstrap.cc:492] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: No bootstrap required, opened a new log
I20260808 10:39:34.553112 4182 ts_tablet_manager.cc:1397] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 10:39:34.553280 4180 tablet_bootstrap.cc:654] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.553273 4188 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.553401 4188 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:34.553546 4182 raft_consensus.cc:348] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.553653 4182 raft_consensus.cc:374] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.553681 4182 raft_consensus.cc:729] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1cbf39e53d44fa9aa713b09fd4ebf8, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.553789 4182 consensus_queue.cc:260] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.554032 4182 ts_tablet_manager.cc:1428] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:34.554008 4190 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.554128 4190 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:34.555033 4180 tablet_bootstrap.cc:492] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: No bootstrap required, opened a new log
I20260808 10:39:34.555145 4180 ts_tablet_manager.cc:1397] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:34.555667 4180 raft_consensus.cc:348] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.555794 4180 raft_consensus.cc:374] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.555840 4180 raft_consensus.cc:729] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.556000 4180 consensus_queue.cc:260] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.556300 4180 ts_tablet_manager.cc:1428] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:39:34.556265 4187 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.556389 4187 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 10:39:34.578097 3997 tablet.cc:2369] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:34.588613 3851 tablet.cc:2369] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:34.675928 4188 raft_consensus.cc:482] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:34.676095 4188 raft_consensus.cc:504] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.676664 4188 leader_election.cc:288] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.677162 3800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b969f5cdaa4a99b430e4447a7fdd70" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add1dfb4998b43d784117053a50d3cd8" is_pre_election: true
I20260808 10:39:34.677201 4100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b969f5cdaa4a99b430e4447a7fdd70" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" is_pre_election: true
I20260808 10:39:34.677361 4100 raft_consensus.cc:2454] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 0.
I20260808 10:39:34.677366 3800 raft_consensus.cc:2454] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 0.
I20260808 10:39:34.677747 3877 leader_election.cc:302] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cbf39e53d44fa9aa713b09fd4ebf8, a23777707f4240a29bcf5f0e2f25eba0; no voters:
I20260808 10:39:34.677997 4188 raft_consensus.cc:2781] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:34.678092 4188 raft_consensus.cc:482] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:34.678160 4188 raft_consensus.cc:3037] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.679279 4188 raft_consensus.cc:504] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.679683 4188 leader_election.cc:288] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 election: Requested vote from peers add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.680101 3800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b969f5cdaa4a99b430e4447a7fdd70" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add1dfb4998b43d784117053a50d3cd8"
I20260808 10:39:34.680104 4100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b969f5cdaa4a99b430e4447a7fdd70" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
I20260808 10:39:34.680236 3800 raft_consensus.cc:3037] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.680236 4100 raft_consensus.cc:3037] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.681430 4100 raft_consensus.cc:2454] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 1.
I20260808 10:39:34.681430 3800 raft_consensus.cc:2454] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 1.
I20260808 10:39:34.681788 3877 leader_election.cc:302] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cbf39e53d44fa9aa713b09fd4ebf8, a23777707f4240a29bcf5f0e2f25eba0; no voters:
I20260808 10:39:34.682024 4188 raft_consensus.cc:2781] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:34.682150 4188 raft_consensus.cc:686] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 LEADER]: Becoming Leader. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Running, Role: LEADER
I20260808 10:39:34.682325 4188 consensus_queue.cc:237] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.683045 4192 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: TxnStatusTablet state changed. Reason: New leader a23777707f4240a29bcf5f0e2f25eba0. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.683178 4192 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:39:34.683462 4205 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:39:34.683730 4205 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:39:34.684264 3632 catalog_manager.cc:5495] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:34.717844 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.713331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.1.194.66:38509)
0808 10:39:34.713462 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.713466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.713614 (+ 148us) server_negotiation.cc:407] Connection header received
0808 10:39:34.713810 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.713812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.713867 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.713952 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:34.714259 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.714766 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.716240 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.716479 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.716612 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.716945 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.717372 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.717458 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.717656 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.717712 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:39:34.717856 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.713225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38509 (local address 127.0.0.1:48472)
0808 10:39:34.713518 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.713535 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.713661 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.714008 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.714012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.714020 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:34.714142 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.714149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.714887 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.714891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.716102 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.716115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.716616 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.716620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.716736 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.716795 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.717505 (+ 710us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.717524 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.717602 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.717702 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 10:39:34.723789 4100 raft_consensus.cc:1261] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Refusing update from remote peer a23777707f4240a29bcf5f0e2f25eba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:34.723789 3800 raft_consensus.cc:1261] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Refusing update from remote peer a23777707f4240a29bcf5f0e2f25eba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:34.724570 4192 consensus_queue.cc:1035] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:34.724771 4188 consensus_queue.cc:1035] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:34.728349 4190 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: TxnStatusTablet state changed. Reason: New leader a23777707f4240a29bcf5f0e2f25eba0. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.728488 4190 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:34.728777 4187 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: TxnStatusTablet state changed. Reason: New leader a23777707f4240a29bcf5f0e2f25eba0. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.728902 4187 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:34.733055 4192 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.733182 4192 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:39:34.733970 4190 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.734078 4190 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:34.734165 4188 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.734275 4188 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P a23777707f4240a29bcf5f0e2f25eba0: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:39:34.735502 4209 mvcc.cc:205] Tried to move back new op lower bound from 7316216114061160448 to 7316216113900126208. Current Snapshot: MvccSnapshot[applied={T|T < 7316216114061160448}]
I20260808 10:39:34.736086 4187 tablet_replica.cc:439] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } } }
I20260808 10:39:34.736214 4187 tablet_replica.cc:442] T c1b969f5cdaa4a99b430e4447a7fdd70 P add1dfb4998b43d784117053a50d3cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:39:34.737931 4208 mvcc.cc:205] Tried to move back new op lower bound from 7316216114061160448 to 7316216113900126208. Current Snapshot: MvccSnapshot[applied={T|T < 7316216114061160448 or (T in {7316216114061160448})}]
I20260808 10:39:34.738153 4210 mvcc.cc:205] Tried to move back new op lower bound from 7316216114061160448 to 7316216113900126208. Current Snapshot: MvccSnapshot[applied={T|T < 7316216114061160448}]
I20260808 10:39:34.745041 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.741711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48476 (local address 127.1.194.66:38509)
0808 10:39:34.741830 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.741834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.741849 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:39:34.741961 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.741964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.742012 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.742095 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:34.742310 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.743102 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.743986 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.744322 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.744424 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.744636 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.744719 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.744785 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.744866 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.744914 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:39:34.755909 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.752322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:39335 (local address 127.1.194.66:38509)
0808 10:39:34.752449 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.752453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.752683 (+ 230us) server_negotiation.cc:407] Connection header received
0808 10:39:34.752794 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.752796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.752835 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.752894 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:34.753150 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.753693 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.754895 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.755098 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.755176 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.755501 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.755565 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.755616 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.755735 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.755799 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":46}
I20260808 10:39:34.756532 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.752220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38509 (local address 127.1.194.66:39335)
0808 10:39:34.752596 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.752608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.752708 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.752923 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.752926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.752933 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:34.753051 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.753057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.753786 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.753789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.754774 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.754784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.755202 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.755206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.755302 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.755351 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.755644 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.755650 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.755715 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.755807 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260808 10:39:34.764269 4100 raft_consensus.cc:1261] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Refusing update from remote peer a23777707f4240a29bcf5f0e2f25eba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:34.764307 3800 raft_consensus.cc:1261] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Refusing update from remote peer a23777707f4240a29bcf5f0e2f25eba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:34.764667 4188 consensus_queue.cc:1035] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:34.764823 4192 consensus_queue.cc:1035] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:34.804208 3632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:34.804404 3632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:34.806954 3632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Sending DeleteTablet for 3 replicas of tablet 05972afcb58046f88990d1fc904a6082
I20260808 10:39:34.807638 3927 tablet_service.cc:1514] Processing DeleteTablet for tablet 05972afcb58046f88990d1fc904a6082 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:34 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:48456
I20260808 10:39:34.807773 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet 05972afcb58046f88990d1fc904a6082 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:34 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:36194
I20260808 10:39:34.808388 4227 tablet_replica.cc:330] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: stopping tablet replica
I20260808 10:39:34.808379 4078 tablet_service.cc:1514] Processing DeleteTablet for tablet 05972afcb58046f88990d1fc904a6082 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:34 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:56754
I20260808 10:39:34.808384 3632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:39:34.808773 4229 tablet_replica.cc:330] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: stopping tablet replica
W20260808 10:39:34.808782 3632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:34.808982 4227 raft_consensus.cc:2229] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:34.809084 4229 raft_consensus.cc:2229] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:34.809314 4229 raft_consensus.cc:2258] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:34.809501 4227 raft_consensus.cc:2258] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:34.810084 4228 tablet_replica.cc:330] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: stopping tablet replica
I20260808 10:39:34.810369 4228 raft_consensus.cc:2229] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:34.810626 4228 raft_consensus.cc:2258] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:34.811072 4229 ts_tablet_manager.cc:1905] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:34.811985 4227 ts_tablet_manager.cc:1905] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:34.814791 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 4313d4c09269435b9415837551a0341e (DEFAULT_TABLE table=test-table [id=78b8081be59642f195d8a3de9c0d7def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:34.814914 3927 tablet_service.cc:1467] Processing CreateTablet for tablet 4313d4c09269435b9415837551a0341e (DEFAULT_TABLE table=test-table [id=78b8081be59642f195d8a3de9c0d7def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:34.815104 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4313d4c09269435b9415837551a0341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.815197 3927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4313d4c09269435b9415837551a0341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.816829 4228 ts_tablet_manager.cc:1905] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:34.817587 4183 tablet_bootstrap.cc:492] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Bootstrap starting.
I20260808 10:39:34.817696 4180 tablet_bootstrap.cc:492] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Bootstrap starting.
I20260808 10:39:34.817943 4078 tablet_service.cc:1467] Processing CreateTablet for tablet 4313d4c09269435b9415837551a0341e (DEFAULT_TABLE table=test-table [id=78b8081be59642f195d8a3de9c0d7def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:34.818207 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4313d4c09269435b9415837551a0341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:34.818562 4183 tablet_bootstrap.cc:654] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.818562 4180 tablet_bootstrap.cc:654] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.820262 4227 ts_tablet_manager.cc:1918] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:34.820447 4227 log.cc:1196] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/05972afcb58046f88990d1fc904a6082
I20260808 10:39:34.820577 4182 tablet_bootstrap.cc:492] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Bootstrap starting.
I20260808 10:39:34.820907 4227 ts_tablet_manager.cc:1939] T 05972afcb58046f88990d1fc904a6082 P a23777707f4240a29bcf5f0e2f25eba0: Deleting consensus metadata
I20260808 10:39:34.820281 4228 ts_tablet_manager.cc:1918] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:34.821105 4228 log.cc:1196] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/05972afcb58046f88990d1fc904a6082
I20260808 10:39:34.821439 4228 ts_tablet_manager.cc:1939] T 05972afcb58046f88990d1fc904a6082 P add1dfb4998b43d784117053a50d3cd8: Deleting consensus metadata
I20260808 10:39:34.829999 3618 catalog_manager.cc:4841] TS add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707): tablet 05972afcb58046f88990d1fc904a6082 (table test-table [id=8addf33fc9ca427880b3af2b842ed559]) successfully deleted
I20260808 10:39:34.830106 4180 tablet_bootstrap.cc:492] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: No bootstrap required, opened a new log
I20260808 10:39:34.830217 4180 ts_tablet_manager.cc:1397] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20260808 10:39:34.830711 4180 raft_consensus.cc:348] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.830852 4180 raft_consensus.cc:374] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.830894 4180 raft_consensus.cc:729] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.831033 4180 consensus_queue.cc:260] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.831315 4180 ts_tablet_manager.cc:1428] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:34.831532 3618 catalog_manager.cc:4841] TS a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509): tablet 05972afcb58046f88990d1fc904a6082 (table test-table [id=8addf33fc9ca427880b3af2b842ed559]) successfully deleted
I20260808 10:39:34.831630 4182 tablet_bootstrap.cc:654] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:34.832219 4183 tablet_bootstrap.cc:492] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: No bootstrap required, opened a new log
I20260808 10:39:34.832261 4229 ts_tablet_manager.cc:1918] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:34.832345 4183 ts_tablet_manager.cc:1397] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260808 10:39:34.832363 4229 log.cc:1196] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/05972afcb58046f88990d1fc904a6082
I20260808 10:39:34.832897 4182 tablet_bootstrap.cc:492] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: No bootstrap required, opened a new log
I20260808 10:39:34.832980 4182 ts_tablet_manager.cc:1397] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260808 10:39:34.832930 4183 raft_consensus.cc:348] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.833082 4183 raft_consensus.cc:374] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.833129 4183 raft_consensus.cc:729] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.833269 4183 consensus_queue.cc:260] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.833400 4182 raft_consensus.cc:348] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.833531 4182 raft_consensus.cc:374] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:34.833587 4182 raft_consensus.cc:729] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1cbf39e53d44fa9aa713b09fd4ebf8, State: Initialized, Role: FOLLOWER
I20260808 10:39:34.833596 4183 ts_tablet_manager.cc:1428] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:39:34.833726 4182 consensus_queue.cc:260] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.833973 4182 ts_tablet_manager.cc:1428] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:34.834579 4229 ts_tablet_manager.cc:1939] T 05972afcb58046f88990d1fc904a6082 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting consensus metadata
I20260808 10:39:34.835654 3617 catalog_manager.cc:4841] TS 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507): tablet 05972afcb58046f88990d1fc904a6082 (table test-table [id=8addf33fc9ca427880b3af2b842ed559]) successfully deleted
I20260808 10:39:34.892899 4187 raft_consensus.cc:482] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:34.893077 4187 raft_consensus.cc:504] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.894274 4187 leader_election.cc:288] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.898756 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.894816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:56143 (local address 127.1.194.67:42507)
0808 10:39:34.894940 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.894945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.895131 (+ 186us) server_negotiation.cc:407] Connection header received
0808 10:39:34.895346 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.895350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.895403 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.895470 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.895758 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.896482 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.897715 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.898183 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.898304 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.898355 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.898529 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.898610 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:39:34.898737 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.894509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:42507 (local address 127.1.194.65:56143)
0808 10:39:34.895046 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.895068 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.895190 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.895537 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.895540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.895558 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.895659 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.895665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.896655 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.896657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.897591 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.897601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.898304 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.898307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.898396 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.898421 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.898503 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.898570 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":165,"thread_start_us":111,"threads_started":1}
I20260808 10:39:34.899330 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.894506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38509 (local address 127.1.194.65:50237)
0808 10:39:34.894996 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.895012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.895143 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.895409 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.895412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.895433 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:34.895555 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.895564 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.896587 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.896591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.897968 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.897979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.898933 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.898937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.899025 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.899049 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.899123 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.899183 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":27,"thread_start_us":80,"threads_started":1}
I20260808 10:39:34.899356 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.894691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:50237 (local address 127.1.194.66:38509)
0808 10:39:34.894824 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.894829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.895113 (+ 284us) server_negotiation.cc:407] Connection header received
0808 10:39:34.895258 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.895261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.895314 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.895399 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:34.895682 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.896466 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.898109 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.898679 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.898803 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.898855 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.899163 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.899239 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:39:34.899585 4101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4313d4c09269435b9415837551a0341e" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" is_pre_election: true
I20260808 10:39:34.899711 4101 raft_consensus.cc:2454] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 0.
I20260808 10:39:34.899941 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4313d4c09269435b9415837551a0341e" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23777707f4240a29bcf5f0e2f25eba0" is_pre_election: true
I20260808 10:39:34.900141 3726 leader_election.cc:302] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cbf39e53d44fa9aa713b09fd4ebf8, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:34.900193 3949 raft_consensus.cc:2454] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 0.
I20260808 10:39:34.900406 4187 raft_consensus.cc:2781] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:34.900480 4187 raft_consensus.cc:482] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:34.900543 4187 raft_consensus.cc:3037] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.902261 4187 raft_consensus.cc:504] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.902633 4187 leader_election.cc:288] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 election: Requested vote from peers a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:34.902967 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4313d4c09269435b9415837551a0341e" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23777707f4240a29bcf5f0e2f25eba0"
I20260808 10:39:34.903122 3949 raft_consensus.cc:3037] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.903107 4101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4313d4c09269435b9415837551a0341e" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
I20260808 10:39:34.903228 4101 raft_consensus.cc:3037] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:34.904480 3949 raft_consensus.cc:2454] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 1.
I20260808 10:39:34.904480 4101 raft_consensus.cc:2454] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 1.
I20260808 10:39:34.904844 3727 leader_election.cc:302] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23777707f4240a29bcf5f0e2f25eba0, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:34.905185 4187 raft_consensus.cc:2781] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:34.905390 4187 raft_consensus.cc:686] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 1 LEADER]: Becoming Leader. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Running, Role: LEADER
I20260808 10:39:34.905681 4187 consensus_queue.cc:237] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:34.908159 3631 catalog_manager.cc:5495] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to add1dfb4998b43d784117053a50d3cd8 (127.1.194.65). New cstate: current_term: 1 leader_uuid: "add1dfb4998b43d784117053a50d3cd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:34.935168 4179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.931510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.1.194.65:34707)
0808 10:39:34.931626 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.931632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.931665 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:39:34.931869 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.931872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.931923 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.932035 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:34.932294 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.932804 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.933723 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.933944 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.934057 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.934331 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.934778 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.934906 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.934973 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.935026 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:39:34.945907 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.942632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:41175 (local address 127.1.194.66:38509)
0808 10:39:34.942766 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.942771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.942892 (+ 121us) server_negotiation.cc:407] Connection header received
0808 10:39:34.943058 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.943061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.943111 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.943193 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:34.943457 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.943949 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.944809 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.945024 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.945134 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.945426 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.945535 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.945625 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.945712 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.945759 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:39:34.945922 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.942468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38509 (local address 127.1.194.65:41175)
0808 10:39:34.942793 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.942816 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.942937 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.943206 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.943209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.943217 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:34.943344 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.943349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.944071 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.944075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.944687 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.944695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.945140 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.945143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.945241 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.945281 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.945631 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.945640 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.945703 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.945780 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
I20260808 10:39:34.952847 4101 raft_consensus.cc:1261] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Refusing update from remote peer add1dfb4998b43d784117053a50d3cd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:34.953002 3949 raft_consensus.cc:1261] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Refusing update from remote peer add1dfb4998b43d784117053a50d3cd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:34.953341 4187 consensus_queue.cc:1035] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:34.953578 4236 consensus_queue.cc:1035] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:34.972453 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.968420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:34707 (local address 127.1.194.66:47903)
0808 10:39:34.968622 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:34.968638 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:34.968715 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:34.969099 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:34.969102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:34.969109 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:39:34.969201 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.969206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.970123 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.970126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.970815 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:34.970828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.971391 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.971395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:34.971499 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.971531 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.971884 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.971891 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:39:34.971989 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:39:34.972089 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 10:39:34.972795 4179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:34.968495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:47903 (local address 127.1.194.65:34707)
0808 10:39:34.968847 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:34.968851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:34.968862 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:39:34.968900 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:34.968903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:34.968943 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:34.969003 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:34.969311 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.969997 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.970933 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:34.971253 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:34.971379 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:34.971690 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:34.971766 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:34.971834 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:34.972582 (+ 748us) server_negotiation.cc:299] Negotiation successful
0808 10:39:34.972644 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
W20260808 10:39:34.973660 4209 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:39:36.537213 3631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:36.537374 3631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:36.540117 3631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Sending DeleteTablet for 3 replicas of tablet 4313d4c09269435b9415837551a0341e
I20260808 10:39:36.540771 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet 4313d4c09269435b9415837551a0341e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:36194
I20260808 10:39:36.540818 3928 tablet_service.cc:1514] Processing DeleteTablet for tablet 4313d4c09269435b9415837551a0341e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:48456
I20260808 10:39:36.540951 4078 tablet_service.cc:1514] Processing DeleteTablet for tablet 4313d4c09269435b9415837551a0341e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:56754
I20260808 10:39:36.541203 4269 tablet_replica.cc:330] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: stopping tablet replica
I20260808 10:39:36.541368 4269 raft_consensus.cc:2229] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:36.541556 4269 raft_consensus.cc:2258] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:36.541836 4270 tablet_replica.cc:330] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: stopping tablet replica
I20260808 10:39:36.541988 4270 raft_consensus.cc:2229] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:36.542366 4270 raft_consensus.cc:2258] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:36.543090 4270 ts_tablet_manager.cc:1905] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:36.543090 4269 ts_tablet_manager.cc:1905] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:36.544840 4271 tablet_replica.cc:330] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: stopping tablet replica
I20260808 10:39:36.545006 4271 raft_consensus.cc:2229] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:36.545140 4271 raft_consensus.cc:2258] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:36.546211 4269 ts_tablet_manager.cc:1918] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:39:36.546240 4271 ts_tablet_manager.cc:1905] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:36.546309 4269 log.cc:1196] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/4313d4c09269435b9415837551a0341e
I20260808 10:39:36.546618 4269 ts_tablet_manager.cc:1939] T 4313d4c09269435b9415837551a0341e P a23777707f4240a29bcf5f0e2f25eba0: Deleting consensus metadata
I20260808 10:39:36.546926 4270 ts_tablet_manager.cc:1918] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:39:36.547079 4270 log.cc:1196] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/4313d4c09269435b9415837551a0341e
I20260808 10:39:36.547465 4270 ts_tablet_manager.cc:1939] T 4313d4c09269435b9415837551a0341e P add1dfb4998b43d784117053a50d3cd8: Deleting consensus metadata
I20260808 10:39:36.547550 3618 catalog_manager.cc:4841] TS a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509): tablet 4313d4c09269435b9415837551a0341e (table test-table [id=78b8081be59642f195d8a3de9c0d7def]) successfully deleted
I20260808 10:39:36.548236 3618 catalog_manager.cc:4841] TS add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707): tablet 4313d4c09269435b9415837551a0341e (table test-table [id=78b8081be59642f195d8a3de9c0d7def]) successfully deleted
I20260808 10:39:36.548794 4271 ts_tablet_manager.cc:1918] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:39:36.548867 4271 log.cc:1196] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/4313d4c09269435b9415837551a0341e
I20260808 10:39:36.549098 4271 ts_tablet_manager.cc:1939] T 4313d4c09269435b9415837551a0341e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting consensus metadata
I20260808 10:39:36.550266 3617 catalog_manager.cc:4841] TS 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507): tablet 4313d4c09269435b9415837551a0341e (table test-table [id=78b8081be59642f195d8a3de9c0d7def]) successfully deleted
Aug 08 10:39:36 dist-test-slave-ss4n krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:39:36 dist-test-slave-ss4n krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:36.576079 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:36.566952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.1.194.126:46195)
0808 10:39:36.567237 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:36.567245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:36.567377 (+ 132us) server_negotiation.cc:407] Connection header received
0808 10:39:36.567563 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:36.567571 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:36.567615 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:36.567713 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:36.568469 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:36.569026 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:36.569940 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:36.570190 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:36.570327 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:36.572862 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:36.572865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:36.572867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:36.572891 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:36.574580 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:36.574967 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:36.574969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:36.574971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:36.575048 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:36.575289 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:36.575292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:36.575293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:36.575459 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:36.575557 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:36.575746 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 10:39:36.575846 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":109,"threads_started":1}
I20260808 10:39:36.578775 3631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:36.579286 3631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:36.584540 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3f161b16b849c595be1aa0bc704b2e (DEFAULT_TABLE table=test-table [id=b232a5c0206847a49152a05c400198a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:36.584570 3928 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3f161b16b849c595be1aa0bc704b2e (DEFAULT_TABLE table=test-table [id=b232a5c0206847a49152a05c400198a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:36.584823 3928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3f161b16b849c595be1aa0bc704b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:36.584857 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3f161b16b849c595be1aa0bc704b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:36.587060 4280 tablet_bootstrap.cc:492] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Bootstrap starting.
I20260808 10:39:36.588141 4280 tablet_bootstrap.cc:654] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:36.588454 4078 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3f161b16b849c595be1aa0bc704b2e (DEFAULT_TABLE table=test-table [id=b232a5c0206847a49152a05c400198a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:36.588773 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3f161b16b849c595be1aa0bc704b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:36.589629 4280 tablet_bootstrap.cc:492] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: No bootstrap required, opened a new log
I20260808 10:39:36.589758 4280 ts_tablet_manager.cc:1397] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:36.590256 4280 raft_consensus.cc:348] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.590442 4280 raft_consensus.cc:374] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:36.590497 4280 raft_consensus.cc:729] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Initialized, Role: FOLLOWER
I20260808 10:39:36.590694 4280 consensus_queue.cc:260] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.591055 4280 ts_tablet_manager.cc:1428] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:36.591362 4282 tablet_bootstrap.cc:492] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Bootstrap starting.
I20260808 10:39:36.592003 4283 tablet_bootstrap.cc:492] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Bootstrap starting.
I20260808 10:39:36.592458 4282 tablet_bootstrap.cc:654] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:36.593536 4283 tablet_bootstrap.cc:654] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:36.593915 4282 tablet_bootstrap.cc:492] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: No bootstrap required, opened a new log
I20260808 10:39:36.594008 4282 ts_tablet_manager.cc:1397] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:36.594522 4282 raft_consensus.cc:348] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.594671 4282 raft_consensus.cc:374] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:36.594712 4282 raft_consensus.cc:729] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1cbf39e53d44fa9aa713b09fd4ebf8, State: Initialized, Role: FOLLOWER
I20260808 10:39:36.594851 4282 consensus_queue.cc:260] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.595119 4283 tablet_bootstrap.cc:492] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: No bootstrap required, opened a new log
I20260808 10:39:36.595239 4283 ts_tablet_manager.cc:1397] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:36.595258 4282 ts_tablet_manager.cc:1428] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:39:36.595717 4283 raft_consensus.cc:348] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.595888 4283 raft_consensus.cc:374] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:36.595942 4283 raft_consensus.cc:729] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Initialized, Role: FOLLOWER
I20260808 10:39:36.596101 4283 consensus_queue.cc:260] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.596479 4283 ts_tablet_manager.cc:1428] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:39:36.646051 4188 raft_consensus.cc:482] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:36.646224 4188 raft_consensus.cc:504] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.646698 4188 leader_election.cc:288] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:36.647140 4101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f161b16b849c595be1aa0bc704b2e" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" is_pre_election: true
I20260808 10:39:36.647141 3800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f161b16b849c595be1aa0bc704b2e" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add1dfb4998b43d784117053a50d3cd8" is_pre_election: true
I20260808 10:39:36.647332 3800 raft_consensus.cc:2454] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 0.
I20260808 10:39:36.647336 4101 raft_consensus.cc:2454] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 0.
I20260808 10:39:36.647653 3878 leader_election.cc:302] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23777707f4240a29bcf5f0e2f25eba0, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:36.647850 4188 raft_consensus.cc:2781] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:36.647910 4188 raft_consensus.cc:482] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:36.647958 4188 raft_consensus.cc:3037] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:36.648893 4188 raft_consensus.cc:504] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.649224 4188 leader_election.cc:288] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 election: Requested vote from peers add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:36.649566 3800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f161b16b849c595be1aa0bc704b2e" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add1dfb4998b43d784117053a50d3cd8"
I20260808 10:39:36.649677 3800 raft_consensus.cc:3037] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:36.649773 4101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f161b16b849c595be1aa0bc704b2e" candidate_uuid: "a23777707f4240a29bcf5f0e2f25eba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
I20260808 10:39:36.649951 4101 raft_consensus.cc:3037] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:36.650750 3800 raft_consensus.cc:2454] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 1.
I20260808 10:39:36.651064 3878 leader_election.cc:302] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23777707f4240a29bcf5f0e2f25eba0, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:36.651266 4101 raft_consensus.cc:2454] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23777707f4240a29bcf5f0e2f25eba0 in term 1.
I20260808 10:39:36.651300 4188 raft_consensus.cc:2781] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:36.651432 4188 raft_consensus.cc:686] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 LEADER]: Becoming Leader. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Running, Role: LEADER
I20260808 10:39:36.651594 4188 consensus_queue.cc:237] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.653301 3632 catalog_manager.cc:5495] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "a23777707f4240a29bcf5f0e2f25eba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:36.675551 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:36.672110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.1.194.66:38509)
0808 10:39:36.672376 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:36.672383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:36.672394 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:39:36.672432 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:36.672434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:36.672474 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:36.672597 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:36.672863 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:36.673452 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:36.674462 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:36.674706 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:36.674788 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:36.675017 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:36.675099 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:36.675154 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:36.675315 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:39:36.675360 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
I20260808 10:39:36.681635 3800 raft_consensus.cc:1261] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Refusing update from remote peer a23777707f4240a29bcf5f0e2f25eba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:36.681699 4101 raft_consensus.cc:1261] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Refusing update from remote peer a23777707f4240a29bcf5f0e2f25eba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:36.682019 4247 consensus_queue.cc:1035] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:36.682184 4188 consensus_queue.cc:1035] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:36.711037 3632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:36.711199 3632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:36.713203 3632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Sending DeleteTablet for 3 replicas of tablet 2b3f161b16b849c595be1aa0bc704b2e
I20260808 10:39:36.713762 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3f161b16b849c595be1aa0bc704b2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:36194
I20260808 10:39:36.713945 4270 tablet_replica.cc:330] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: stopping tablet replica
I20260808 10:39:36.713912 3928 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3f161b16b849c595be1aa0bc704b2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:48456
I20260808 10:39:36.713996 4078 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3f161b16b849c595be1aa0bc704b2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:56754
I20260808 10:39:36.714066 4270 raft_consensus.cc:2229] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:36.714097 4269 tablet_replica.cc:330] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: stopping tablet replica
I20260808 10:39:36.714182 4271 tablet_replica.cc:330] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: stopping tablet replica
I20260808 10:39:36.714187 4270 raft_consensus.cc:2258] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:36.714246 4269 raft_consensus.cc:2229] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:36.714336 4271 raft_consensus.cc:2229] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:36.714461 4271 raft_consensus.cc:2258] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:36.714462 4269 raft_consensus.cc:2258] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:36.714944 4270 ts_tablet_manager.cc:1905] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:36.715619 3632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:36.716084 3632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:36.717188 4270 ts_tablet_manager.cc:1918] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:36.717268 4270 log.cc:1196] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/2b3f161b16b849c595be1aa0bc704b2e
I20260808 10:39:36.717597 4270 ts_tablet_manager.cc:1939] T 2b3f161b16b849c595be1aa0bc704b2e P add1dfb4998b43d784117053a50d3cd8: Deleting consensus metadata
I20260808 10:39:36.717736 4269 ts_tablet_manager.cc:1905] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:36.717967 4271 ts_tablet_manager.cc:1905] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:36.718423 3618 catalog_manager.cc:4841] TS add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707): tablet 2b3f161b16b849c595be1aa0bc704b2e (table test-table [id=b232a5c0206847a49152a05c400198a4]) successfully deleted
I20260808 10:39:36.720504 4269 ts_tablet_manager.cc:1918] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:36.720501 4271 ts_tablet_manager.cc:1918] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:39:36.720578 4269 log.cc:1196] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/2b3f161b16b849c595be1aa0bc704b2e
I20260808 10:39:36.720578 4271 log.cc:1196] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/2b3f161b16b849c595be1aa0bc704b2e
I20260808 10:39:36.720862 4271 ts_tablet_manager.cc:1939] T 2b3f161b16b849c595be1aa0bc704b2e P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting consensus metadata
I20260808 10:39:36.720862 4269 ts_tablet_manager.cc:1939] T 2b3f161b16b849c595be1aa0bc704b2e P a23777707f4240a29bcf5f0e2f25eba0: Deleting consensus metadata
I20260808 10:39:36.721763 3618 catalog_manager.cc:4841] TS a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509): tablet 2b3f161b16b849c595be1aa0bc704b2e (table test-table [id=b232a5c0206847a49152a05c400198a4]) successfully deleted
I20260808 10:39:36.721763 3617 catalog_manager.cc:4841] TS 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507): tablet 2b3f161b16b849c595be1aa0bc704b2e (table test-table [id=b232a5c0206847a49152a05c400198a4]) successfully deleted
I20260808 10:39:36.724581 4078 tablet_service.cc:1467] Processing CreateTablet for tablet df73da97a9994c4da35544b3554d9be1 (DEFAULT_TABLE table=test-table [id=8df88660aa154614a3891f600b72bbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:36.724651 3928 tablet_service.cc:1467] Processing CreateTablet for tablet df73da97a9994c4da35544b3554d9be1 (DEFAULT_TABLE table=test-table [id=8df88660aa154614a3891f600b72bbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:36.724728 3779 tablet_service.cc:1467] Processing CreateTablet for tablet df73da97a9994c4da35544b3554d9be1 (DEFAULT_TABLE table=test-table [id=8df88660aa154614a3891f600b72bbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:36.724841 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df73da97a9994c4da35544b3554d9be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:36.724960 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df73da97a9994c4da35544b3554d9be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:36.724951 3928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df73da97a9994c4da35544b3554d9be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:36.726959 4280 tablet_bootstrap.cc:492] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Bootstrap starting.
I20260808 10:39:36.727254 4282 tablet_bootstrap.cc:492] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Bootstrap starting.
I20260808 10:39:36.727918 4280 tablet_bootstrap.cc:654] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:36.728271 4282 tablet_bootstrap.cc:654] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:36.729018 4283 tablet_bootstrap.cc:492] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8: Bootstrap starting.
I20260808 10:39:36.729161 4280 tablet_bootstrap.cc:492] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: No bootstrap required, opened a new log
I20260808 10:39:36.729270 4280 ts_tablet_manager.cc:1397] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:36.729736 4280 raft_consensus.cc:348] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.729874 4280 raft_consensus.cc:374] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:36.729885 4283 tablet_bootstrap.cc:654] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:36.729911 4280 raft_consensus.cc:729] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23777707f4240a29bcf5f0e2f25eba0, State: Initialized, Role: FOLLOWER
I20260808 10:39:36.730074 4280 consensus_queue.cc:260] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.730427 4280 ts_tablet_manager.cc:1428] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:39:36.730752 4282 tablet_bootstrap.cc:492] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: No bootstrap required, opened a new log
I20260808 10:39:36.730844 4282 ts_tablet_manager.cc:1397] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:39:36.731168 4283 tablet_bootstrap.cc:492] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8: No bootstrap required, opened a new log
I20260808 10:39:36.731282 4283 ts_tablet_manager.cc:1397] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:36.731287 4282 raft_consensus.cc:348] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.731391 4282 raft_consensus.cc:374] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:36.731451 4282 raft_consensus.cc:729] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1cbf39e53d44fa9aa713b09fd4ebf8, State: Initialized, Role: FOLLOWER
I20260808 10:39:36.731592 4282 consensus_queue.cc:260] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.731747 4283 raft_consensus.cc:348] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.731870 4283 raft_consensus.cc:374] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:36.731906 4283 raft_consensus.cc:729] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Initialized, Role: FOLLOWER
I20260808 10:39:36.731988 4282 ts_tablet_manager.cc:1428] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:36.732039 4283 consensus_queue.cc:260] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.732319 4283 ts_tablet_manager.cc:1428] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:36.805748 4262 raft_consensus.cc:482] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:36.805912 4262 raft_consensus.cc:504] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.806380 4262 leader_election.cc:288] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:36.806810 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df73da97a9994c4da35544b3554d9be1" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23777707f4240a29bcf5f0e2f25eba0" is_pre_election: true
I20260808 10:39:36.806810 4101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df73da97a9994c4da35544b3554d9be1" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" is_pre_election: true
I20260808 10:39:36.806962 3949 raft_consensus.cc:2454] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 0.
I20260808 10:39:36.806962 4101 raft_consensus.cc:2454] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 0.
I20260808 10:39:36.807324 3726 leader_election.cc:302] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cbf39e53d44fa9aa713b09fd4ebf8, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:36.807535 4262 raft_consensus.cc:2781] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:36.807603 4262 raft_consensus.cc:482] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:36.807632 4262 raft_consensus.cc:3037] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:36.808681 4262 raft_consensus.cc:504] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.809028 4262 leader_election.cc:288] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 election: Requested vote from peers a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509), 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507)
I20260808 10:39:36.809530 4101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df73da97a9994c4da35544b3554d9be1" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8"
I20260808 10:39:36.809624 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df73da97a9994c4da35544b3554d9be1" candidate_uuid: "add1dfb4998b43d784117053a50d3cd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23777707f4240a29bcf5f0e2f25eba0"
I20260808 10:39:36.809696 4101 raft_consensus.cc:3037] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:36.809759 3949 raft_consensus.cc:3037] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:36.810983 3949 raft_consensus.cc:2454] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 1.
I20260808 10:39:36.810995 4101 raft_consensus.cc:2454] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add1dfb4998b43d784117053a50d3cd8 in term 1.
I20260808 10:39:36.811275 3726 leader_election.cc:302] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cbf39e53d44fa9aa713b09fd4ebf8, add1dfb4998b43d784117053a50d3cd8; no voters:
I20260808 10:39:36.811466 4262 raft_consensus.cc:2781] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:36.811578 4262 raft_consensus.cc:686] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [term 1 LEADER]: Becoming Leader. State: Replica: add1dfb4998b43d784117053a50d3cd8, State: Running, Role: LEADER
I20260808 10:39:36.811717 4262 consensus_queue.cc:237] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } }
I20260808 10:39:36.813333 3633 catalog_manager.cc:5495] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to add1dfb4998b43d784117053a50d3cd8 (127.1.194.65). New cstate: current_term: 1 leader_uuid: "add1dfb4998b43d784117053a50d3cd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add1dfb4998b43d784117053a50d3cd8" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 34707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:36.838372 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:36.835080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.1.194.65:34707)
0808 10:39:36.835389 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:36.835398 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:36.835410 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:36.835453 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:36.835455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:36.835492 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:36.835580 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:36.835856 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:36.836387 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:36.837221 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:36.837460 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:36.837596 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:36.837845 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:36.837926 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:36.838008 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:36.838159 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 10:39:36.838224 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":108,"threads_started":1}
I20260808 10:39:36.844302 4101 raft_consensus.cc:1261] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Refusing update from remote peer add1dfb4998b43d784117053a50d3cd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:36.844453 3949 raft_consensus.cc:1261] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Refusing update from remote peer add1dfb4998b43d784117053a50d3cd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:36.844779 4259 consensus_queue.cc:1035] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1cbf39e53d44fa9aa713b09fd4ebf8" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:36.844959 4262 consensus_queue.cc:1035] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23777707f4240a29bcf5f0e2f25eba0" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:39:36.861380 4249 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:39:38.417065 3633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:38.417279 3633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:38.420176 3633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c2ef721ae78489ab9ac40ecba38657b: Sending DeleteTablet for 3 replicas of tablet df73da97a9994c4da35544b3554d9be1
I20260808 10:39:38.420830 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet df73da97a9994c4da35544b3554d9be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:38 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:36194
I20260808 10:39:38.420854 4078 tablet_service.cc:1514] Processing DeleteTablet for tablet df73da97a9994c4da35544b3554d9be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:38 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:56754
I20260808 10:39:38.420856 3928 tablet_service.cc:1514] Processing DeleteTablet for tablet df73da97a9994c4da35544b3554d9be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:38 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:48456
I20260808 10:39:38.421226 4329 tablet_replica.cc:330] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: stopping tablet replica
I20260808 10:39:38.421288 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 3709
I20260808 10:39:38.421295 4326 tablet_replica.cc:330] T df73da97a9994c4da35544b3554d9be1 P add1dfb4998b43d784117053a50d3cd8: stopping tablet replica
I20260808 10:39:38.422474 4329 raft_consensus.cc:2229] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:38.422623 4329 raft_consensus.cc:2258] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:38.423445 4329 ts_tablet_manager.cc:1905] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:38.424270 4328 tablet_replica.cc:330] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: stopping tablet replica
I20260808 10:39:38.424409 4328 raft_consensus.cc:2229] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:38.424522 4328 raft_consensus.cc:2258] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:38.425212 4328 ts_tablet_manager.cc:1905] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:38.426412 4329 ts_tablet_manager.cc:1918] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:39:38.426503 4329 log.cc:1196] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/df73da97a9994c4da35544b3554d9be1
I20260808 10:39:38.426800 4329 ts_tablet_manager.cc:1939] T df73da97a9994c4da35544b3554d9be1 P 1d1cbf39e53d44fa9aa713b09fd4ebf8: Deleting consensus metadata
I20260808 10:39:38.427603 3617 catalog_manager.cc:4841] TS 1d1cbf39e53d44fa9aa713b09fd4ebf8 (127.1.194.67:42507): tablet df73da97a9994c4da35544b3554d9be1 (table test-table [id=8df88660aa154614a3891f600b72bbb5]) successfully deleted
I20260808 10:39:38.427999 4328 ts_tablet_manager.cc:1918] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:39:38.428063 4328 log.cc:1196] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/df73da97a9994c4da35544b3554d9be1
I20260808 10:39:38.428308 4328 ts_tablet_manager.cc:1939] T df73da97a9994c4da35544b3554d9be1 P a23777707f4240a29bcf5f0e2f25eba0: Deleting consensus metadata
I20260808 10:39:38.429193 3618 catalog_manager.cc:4841] TS a23777707f4240a29bcf5f0e2f25eba0 (127.1.194.66:38509): tablet df73da97a9994c4da35544b3554d9be1 (table test-table [id=8df88660aa154614a3891f600b72bbb5]) successfully deleted
I20260808 10:39:38.431300 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 3859
W20260808 10:39:38.431293 3618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:34707 (error 108)
I20260808 10:39:38.435156 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:38.434629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:34707 (local address 127.0.0.1:36218)
0808 10:39:38.434973 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:38.435064 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":102,"threads_started":1}
W20260808 10:39:38.435456 3618 catalog_manager.cc:4584] TS add1dfb4998b43d784117053a50d3cd8 (127.1.194.65:34707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df73da97a9994c4da35544b3554d9be1: Network error: Client connection negotiation failed: client connection to 127.1.194.65:34707: connect: Connection refused (error 111)
I20260808 10:39:38.440140 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 4006
I20260808 10:39:38.446955 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 3599
2026-08-08T10:39:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6361 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:38 dist-test-slave-ss4n krb5kdc[4335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:38 dist-test-slave-ss4n krb5kdc[4335](info): set up 2 sockets
Aug 08 10:39:38 dist-test-slave-ss4n krb5kdc[4335](info): commencing operation
krb5kdc: starting...
W20260808 10:39:39.527336 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:39 dist-test-slave-ss4n krb5kdc[4335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:39.660465 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:39899
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:39899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:39.772452 4349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:39.772712 4349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:39.772789 4349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:39.777112 4349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:39.777190 4349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:39.777218 4349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:39.777242 4349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:39.777272 4349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:39.783329 4349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:39899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:39899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:39.784296 4349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:39.785250 4349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:39.794190 4354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:39.794229 4355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:39.794236 4349 server_base.cc:1034] running on GCE node
W20260808 10:39:39.794600 4357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:39.795158 4349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:39.796327 4349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:39.797566 4349 hybrid_clock.cc:648] HybridClock initialized: now 1786185579797536 us; error 45 us; skew 500 ppm
Aug 08 10:39:39 dist-test-slave-ss4n krb5kdc[4335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:39.805629 4349 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:39.806535 4349 webserver.cc:460] Webserver started at http://127.1.194.126:38557/ using document root <none> and password file <none>
I20260808 10:39:39.807073 4349 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:39.807132 4349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:39.807299 4349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:39.809332 4349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "ffe62696af1e4e8e9c5ea930468606b6"
format_stamp: "Formatted at 2026-08-08 10:39:39 on dist-test-slave-ss4n"
server_key: "66918758cf6e61dc588df2bbf6895283"
server_key_iv: "561b663774758c07ccc088e22f331c5d"
server_key_version: "encryptionkey@0"
I20260808 10:39:39.809942 4349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "ffe62696af1e4e8e9c5ea930468606b6"
format_stamp: "Formatted at 2026-08-08 10:39:39 on dist-test-slave-ss4n"
server_key: "66918758cf6e61dc588df2bbf6895283"
server_key_iv: "561b663774758c07ccc088e22f331c5d"
server_key_version: "encryptionkey@0"
I20260808 10:39:39.813890 4349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:39:39.816231 4365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:39.817651 4349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:39.817778 4349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "ffe62696af1e4e8e9c5ea930468606b6"
format_stamp: "Formatted at 2026-08-08 10:39:39 on dist-test-slave-ss4n"
server_key: "66918758cf6e61dc588df2bbf6895283"
server_key_iv: "561b663774758c07ccc088e22f331c5d"
server_key_version: "encryptionkey@0"
I20260808 10:39:39.817890 4349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:39.836360 4349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:39.839609 4349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:39.839831 4349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:39.847357 4349 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:39899
I20260808 10:39:39.847389 4427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:39899 every 8 connection(s)
I20260808 10:39:39.848366 4349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:39.851217 4428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:39.856556 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6: Bootstrap starting.
I20260808 10:39:39.857897 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 4349
I20260808 10:39:39.858031 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:39.858301 1801 external_mini_cluster.cc:1428] Setting key 4cbbad72e5444bf672a7d891dca378a9
I20260808 10:39:39.859382 4428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:39.860208 4428 log.cc:824] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:39.862289 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6: No bootstrap required, opened a new log
I20260808 10:39:39.866006 4428 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe62696af1e4e8e9c5ea930468606b6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39899 } }
I20260808 10:39:39.866220 4428 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:39.866254 4428 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe62696af1e4e8e9c5ea930468606b6, State: Initialized, Role: FOLLOWER
I20260808 10:39:39.866703 4428 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe62696af1e4e8e9c5ea930468606b6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39899 } }
I20260808 10:39:39.866858 4428 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:39.866945 4428 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:39.867075 4428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 10:39:39 dist-test-slave-ss4n krb5kdc[4335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:39.868270 4428 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe62696af1e4e8e9c5ea930468606b6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39899 } }
I20260808 10:39:39.868626 4428 leader_election.cc:302] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffe62696af1e4e8e9c5ea930468606b6; no voters:
I20260808 10:39:39.869014 4428 leader_election.cc:288] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:39.869132 4433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:39.869357 4433 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [term 1 LEADER]: Becoming Leader. State: Replica: ffe62696af1e4e8e9c5ea930468606b6, State: Running, Role: LEADER
I20260808 10:39:39.869737 4433 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe62696af1e4e8e9c5ea930468606b6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39899 } }
I20260808 10:39:39.870395 4428 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:39.871079 4433 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffe62696af1e4e8e9c5ea930468606b6. Latest consensus state: current_term: 1 leader_uuid: "ffe62696af1e4e8e9c5ea930468606b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe62696af1e4e8e9c5ea930468606b6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39899 } } }
I20260808 10:39:39.871307 4433 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:39.871682 4434 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffe62696af1e4e8e9c5ea930468606b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe62696af1e4e8e9c5ea930468606b6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39899 } } }
I20260808 10:39:39.871786 4434 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:39.873034 4442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:39.874392 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:39.859492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.1.194.126:39899)
0808 10:39:39.859950 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:39.859964 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:39.859998 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:39:39.860818 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:39.860842 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:39.861208 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:39.861578 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:39.862463 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:39.863503 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:39.864363 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:39.864829 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:39.865008 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:39.867934 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:39.867939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:39.867952 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:39.867979 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:39.870203 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:39.870747 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:39.870753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:39.870758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:39.870909 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:39.871528 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:39.871533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:39.871538 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:39.872141 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:39.872434 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:39.873231 (+ 797us) server_negotiation.cc:299] Negotiation successful
0808 10:39:39.873585 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":143,"threads_started":1}
I20260808 10:39:39.876140 4442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:39.881759 4442 catalog_manager.cc:1347] Generated new cluster ID: fe7d6bf10fa34c12b07ef4ea27a27eef
I20260808 10:39:39.881850 4442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:39.911870 4442 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:39.913080 4442 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:39.920939 4442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffe62696af1e4e8e9c5ea930468606b6: Generated new TSK 0
I20260808 10:39:39.922036 4442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:39.956251 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39899
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:40.070271 4455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:40.070528 4455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:40.070570 4455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:40.074838 4455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:40.074890 4455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:40.074947 4455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:40.079968 4455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:40.080892 4455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:40.081957 4455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:40.091866 4464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:40.091868 4461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:40.092048 4462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:40.092253 4455 server_base.cc:1034] running on GCE node
I20260808 10:39:40.092909 4455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:40.093611 4455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:40.094797 4455 hybrid_clock.cc:648] HybridClock initialized: now 1786185580094768 us; error 41 us; skew 500 ppm
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:40.097872 4455 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:40.098784 4455 webserver.cc:460] Webserver started at http://127.1.194.65:39789/ using document root <none> and password file <none>
I20260808 10:39:40.099331 4455 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:40.099421 4455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:40.099653 4455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:40.101707 4455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "c0d00d0106fa4a928855dc1446cf27d8"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "a29777658bba7dde4ba21d694f389716"
server_key_iv: "b2fca030d8217ee81882d3a77fdd36ab"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.102276 4455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "c0d00d0106fa4a928855dc1446cf27d8"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "a29777658bba7dde4ba21d694f389716"
server_key_iv: "b2fca030d8217ee81882d3a77fdd36ab"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.105839 4455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:40.108131 4471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.109293 4455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:40.109432 4455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "c0d00d0106fa4a928855dc1446cf27d8"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "a29777658bba7dde4ba21d694f389716"
server_key_iv: "b2fca030d8217ee81882d3a77fdd36ab"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.109601 4455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:40.140832 4455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:40.143805 4455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:40.144021 4455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:40.144604 4455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:40.145428 4455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:40.145519 4455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.145597 4455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:40.145656 4455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.155174 4455 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:36303
I20260808 10:39:40.155206 4593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:36303 every 8 connection(s)
I20260808 10:39:40.156041 4455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:40.163558 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 4455
I20260808 10:39:40.163707 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:40.163992 1801 external_mini_cluster.cc:1428] Setting key 88bd5d4fa19057f4618837436512bd3c
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:40.170114 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:40.157868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:36449 (local address 127.1.194.126:39899)
0808 10:39:40.158018 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:40.158021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:40.158765 (+ 744us) server_negotiation.cc:407] Connection header received
0808 10:39:40.160105 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:40.160108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:40.160168 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:40.160278 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:40.161432 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.162043 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.163274 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.163540 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.163662 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:40.166284 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:40.166288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:40.166290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:40.166312 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:40.167930 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:40.168482 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:40.168486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:40.168488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:40.168570 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:40.168908 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:40.168913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:40.168915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:40.169101 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:40.169210 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:40.169778 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 10:39:40.169924 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:39:40.171120 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:40.158170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39899 (local address 127.1.194.65:36449)
0808 10:39:40.158607 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:40.158649 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:40.159896 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:40.160443 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:40.160449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:40.160804 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:40.161277 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:40.161291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.162189 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.162192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:40.163141 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:40.163150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.163701 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.163705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:40.163910 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:40.164620 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:40.164635 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:40.166114 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:40.168079 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:40.168084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:40.168098 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:40.168375 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:40.168665 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:40.168670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:40.168672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:40.168810 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:40.169209 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:40.169216 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:40.169459 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 10:39:40.169734 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 10:39:40.170073 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":111,"threads_started":1}
I20260808 10:39:40.172448 4594 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39899
I20260808 10:39:40.172732 4594 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:40.173321 4594 heartbeater.cc:511] Master 127.1.194.126:39899 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:40.175079 4383 ts_manager.cc:194] Registered new tserver with Master: c0d00d0106fa4a928855dc1446cf27d8 (127.1.194.65:36303)
I20260808 10:39:40.176194 4383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:36449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:40.197221 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39899
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:40.307569 4600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:40.307821 4600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:40.307863 4600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:40.311734 4600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:40.311786 4600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:40.311843 4600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:40.317090 4600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:40.318112 4600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:40.319077 4600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:40.328894 4608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:40.328887 4605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:40.328887 4606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:40.329138 4600 server_base.cc:1034] running on GCE node
I20260808 10:39:40.329748 4600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:40.330411 4600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:40.331602 4600 hybrid_clock.cc:648] HybridClock initialized: now 1786185580331567 us; error 52 us; skew 500 ppm
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:40.334756 4600 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:40.335654 4600 webserver.cc:460] Webserver started at http://127.1.194.66:34333/ using document root <none> and password file <none>
I20260808 10:39:40.336206 4600 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:40.336272 4600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:40.336452 4600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:40.338650 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "f6deceb18c8b4b45845ac4e24f91ab7a"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "77e2fd99f349c5c194399e04aa79bbcc"
server_key_iv: "91e0b1bce11f8f41bb3259e7922d0134"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.339206 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "f6deceb18c8b4b45845ac4e24f91ab7a"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "77e2fd99f349c5c194399e04aa79bbcc"
server_key_iv: "91e0b1bce11f8f41bb3259e7922d0134"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.343096 4600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:40.345575 4615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.346860 4600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:40.346973 4600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "f6deceb18c8b4b45845ac4e24f91ab7a"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "77e2fd99f349c5c194399e04aa79bbcc"
server_key_iv: "91e0b1bce11f8f41bb3259e7922d0134"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.347126 4600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:40.366356 4600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:40.370102 4600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:40.370334 4600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:40.371099 4600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:40.372216 4600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:40.372274 4600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.372349 4600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:40.372390 4600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.382138 4600 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:37285
I20260808 10:39:40.382206 4744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:37285 every 8 connection(s)
I20260808 10:39:40.383189 4600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:40.384373 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 4600
I20260808 10:39:40.384485 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:40.384804 1801 external_mini_cluster.cc:1428] Setting key 5dc8d7b3d963efebbe13b42e805391e6
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:40.397763 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:40.385231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:38493 (local address 127.1.194.126:39899)
0808 10:39:40.385399 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:40.385403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:40.386477 (+ 1074us) server_negotiation.cc:407] Connection header received
0808 10:39:40.387796 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:40.387800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:40.387854 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:40.387925 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:40.389025 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.389650 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.390868 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.391110 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.391223 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:40.393829 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:40.393833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:40.393836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:40.393870 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:40.395460 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:40.395966 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:40.395970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:40.395972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:40.396060 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:40.396421 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:40.396424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:40.396426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:40.396712 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:40.396807 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:40.397340 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 10:39:40.397617 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:40.398528 4747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:40.385787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39899 (local address 127.1.194.66:38493)
0808 10:39:40.386334 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:40.386388 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:40.387595 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:40.388088 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:40.388094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:40.388443 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:40.388888 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:40.388902 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.389770 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.389773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:40.390742 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:40.390753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.391261 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.391265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:40.391436 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:40.392137 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:40.392152 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:40.393664 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:40.395580 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:40.395586 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:40.395601 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:40.395862 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:40.396157 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:40.396160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:40.396162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:40.396302 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:40.396819 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:40.396824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:40.397055 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 10:39:40.397218 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 10:39:40.397574 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":167,"threads_started":1}
I20260808 10:39:40.400575 4745 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39899
I20260808 10:39:40.400847 4745 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:40.401363 4745 heartbeater.cc:511] Master 127.1.194.126:39899 requested a full tablet report, sending...
I20260808 10:39:40.402484 4383 ts_manager.cc:194] Registered new tserver with Master: f6deceb18c8b4b45845ac4e24f91ab7a (127.1.194.66:37285)
I20260808 10:39:40.403062 4383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:38493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:40.417333 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39899
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:40.536278 4750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:40.536571 4750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:40.536619 4750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:40.540934 4750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:40.540997 4750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:40.541095 4750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:40.547410 4750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:40.548614 4750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:40.549733 4750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:40.560483 4759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:40.560571 4757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:40.560483 4756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:40.560683 4750 server_base.cc:1034] running on GCE node
I20260808 10:39:40.561268 4750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:40.562002 4750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:40.563191 4750 hybrid_clock.cc:648] HybridClock initialized: now 1786185580563170 us; error 31 us; skew 500 ppm
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:40.566345 4750 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:40.567335 4750 webserver.cc:460] Webserver started at http://127.1.194.67:37629/ using document root <none> and password file <none>
I20260808 10:39:40.567975 4750 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:40.568053 4750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:40.568338 4750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:40.570770 4750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "7d4c4167a3f1490eb7faef43cee91d43"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "113f34f0b10ee54312a344b22175b8f2"
server_key_iv: "9e3ecae17095b558ef99f5258485c463"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.571508 4750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "7d4c4167a3f1490eb7faef43cee91d43"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "113f34f0b10ee54312a344b22175b8f2"
server_key_iv: "9e3ecae17095b558ef99f5258485c463"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.575781 4750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:40.578207 4766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.579392 4750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:40.579526 4750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "7d4c4167a3f1490eb7faef43cee91d43"
format_stamp: "Formatted at 2026-08-08 10:39:40 on dist-test-slave-ss4n"
server_key: "113f34f0b10ee54312a344b22175b8f2"
server_key_iv: "9e3ecae17095b558ef99f5258485c463"
server_key_version: "encryptionkey@0"
I20260808 10:39:40.579635 4750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:40.594229 4750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:40.597699 4750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:40.597906 4750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:40.598511 4750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:40.599344 4750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:40.599414 4750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.599484 4750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:40.599536 4750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:40.608321 4750 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:39507
I20260808 10:39:40.608386 4891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:39507 every 8 connection(s)
I20260808 10:39:40.609256 4750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:40.615389 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 4750
I20260808 10:39:40.615521 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:40.615830 1801 external_mini_cluster.cc:1428] Setting key 3b151eda9b24cf6938896e980b5f92d8
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:40.623001 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:40.611078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:51715 (local address 127.1.194.126:39899)
0808 10:39:40.611214 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:40.611218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:40.612019 (+ 801us) server_negotiation.cc:407] Connection header received
0808 10:39:40.613434 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:40.613437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:40.613527 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:40.613634 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:40.614764 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.615414 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.616706 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.616943 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.617056 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:40.619506 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:40.619510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:40.619513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:40.619546 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:40.620858 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:40.621422 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:40.621426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:40.621428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:40.621575 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:40.621921 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:40.621926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:40.621927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:40.622117 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:40.622248 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:40.622715 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 10:39:40.622836 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:39:40.623811 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:40.611443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39899 (local address 127.1.194.67:51715)
0808 10:39:40.611872 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:40.611913 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:40.613171 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:40.613803 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:40.613811 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:40.614168 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:40.614632 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:40.614644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.615558 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.615561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:40.616583 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:40.616593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:40.617103 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:40.617108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:40.617269 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:40.617892 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:40.617907 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:40.619339 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:40.620989 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:40.620995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:40.621010 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:40.621317 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:40.621680 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:40.621682 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:40.621684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:40.621822 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:40.622245 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:40.622253 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:40.622487 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:39:40.622672 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 10:39:40.623000 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
I20260808 10:39:40.624925 4892 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39899
I20260808 10:39:40.625168 4892 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:40.625697 4892 heartbeater.cc:511] Master 127.1.194.126:39899 requested a full tablet report, sending...
I20260808 10:39:40.626698 4383 ts_manager.cc:194] Registered new tserver with Master: 7d4c4167a3f1490eb7faef43cee91d43 (127.1.194.67:39507)
I20260808 10:39:40.627259 4383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:51715
I20260808 10:39:40.629663 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:39:40.644003 4431 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:40.642703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.1.194.126:39899)
0808 10:39:40.642841 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:40.642844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:40.642878 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:39:40.643014 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:40.643017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:40.643061 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:40.643146 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:40.643833 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43092: BlockingRecv error: recv got EOF from 127.0.0.1:43092 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:39:40.645170 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 4455
I20260808 10:39:40.651754 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 4600
I20260808 10:39:40.658571 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 4750
I20260808 10:39:40.666575 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 4349
2026-08-08T10:39:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2217 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4904](info): set up 2 sockets
Aug 08 10:39:40 dist-test-slave-ss4n krb5kdc[4904](info): commencing operation
krb5kdc: starting...
W20260808 10:39:41.820022 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:41 dist-test-slave-ss4n krb5kdc[4904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:41.953269 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46641
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:46641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:42.071262 4918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:42.071566 4918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:42.071643 4918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:42.075784 4918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:42.075871 4918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:42.075910 4918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:42.075947 4918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:42.075986 4918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:42.081192 4918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:46641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:42.082281 4918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:42.083330 4918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:42.091815 4925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:42.091993 4918 server_base.cc:1034] running on GCE node
W20260808 10:39:42.091931 4924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:42.091815 4928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:42.092743 4918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:42.093904 4918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:42.095077 4918 hybrid_clock.cc:648] HybridClock initialized: now 1786185582095063 us; error 30 us; skew 500 ppm
Aug 08 10:39:42 dist-test-slave-ss4n krb5kdc[4904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185582, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:42.098233 4918 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:42.099125 4918 webserver.cc:460] Webserver started at http://127.1.194.126:42139/ using document root <none> and password file <none>
I20260808 10:39:42.099749 4918 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:42.099840 4918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:42.100080 4918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:42.102226 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "151818fb15694865b7b10fd0bc64c2b0"
format_stamp: "Formatted at 2026-08-08 10:39:42 on dist-test-slave-ss4n"
server_key: "bfbe2a2c42197e8af262ff74f4c8ae89"
server_key_iv: "ca310fd61560d37f5f85f20daa103484"
server_key_version: "encryptionkey@0"
I20260808 10:39:42.102828 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "151818fb15694865b7b10fd0bc64c2b0"
format_stamp: "Formatted at 2026-08-08 10:39:42 on dist-test-slave-ss4n"
server_key: "bfbe2a2c42197e8af262ff74f4c8ae89"
server_key_iv: "ca310fd61560d37f5f85f20daa103484"
server_key_version: "encryptionkey@0"
I20260808 10:39:42.106819 4918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:39:42.109310 4935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:42.110626 4918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:42.110795 4918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "151818fb15694865b7b10fd0bc64c2b0"
format_stamp: "Formatted at 2026-08-08 10:39:42 on dist-test-slave-ss4n"
server_key: "bfbe2a2c42197e8af262ff74f4c8ae89"
server_key_iv: "ca310fd61560d37f5f85f20daa103484"
server_key_version: "encryptionkey@0"
I20260808 10:39:42.110913 4918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:42.128670 4918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:42.132097 4918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:42.132311 4918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:42.140064 4918 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:46641
I20260808 10:39:42.140504 4995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:46641 every 8 connection(s)
I20260808 10:39:42.141451 4918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:42.144344 4996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:42.149816 4996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0: Bootstrap starting.
I20260808 10:39:42.150465 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 4918
I20260808 10:39:42.150564 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:42.150885 1801 external_mini_cluster.cc:1428] Setting key 95940006683354a0d848d55edee284a3
I20260808 10:39:42.152554 4996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:42.153438 4996 log.cc:824] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:42.155635 4996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0: No bootstrap required, opened a new log
I20260808 10:39:42.159597 4996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151818fb15694865b7b10fd0bc64c2b0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46641 } }
I20260808 10:39:42.159839 4996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:42.159924 4996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151818fb15694865b7b10fd0bc64c2b0, State: Initialized, Role: FOLLOWER
Aug 08 10:39:42 dist-test-slave-ss4n krb5kdc[4904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:42.160435 4996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151818fb15694865b7b10fd0bc64c2b0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46641 } }
I20260808 10:39:42.160602 4996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:42.160671 4996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:42.160764 4996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:42.161921 4996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151818fb15694865b7b10fd0bc64c2b0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46641 } }
I20260808 10:39:42.162284 4996 leader_election.cc:302] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 151818fb15694865b7b10fd0bc64c2b0; no voters:
I20260808 10:39:42.162715 4996 leader_election.cc:288] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:42.162791 5004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:42.162966 5004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [term 1 LEADER]: Becoming Leader. State: Replica: 151818fb15694865b7b10fd0bc64c2b0, State: Running, Role: LEADER
I20260808 10:39:42.163245 5004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151818fb15694865b7b10fd0bc64c2b0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46641 } }
I20260808 10:39:42.163975 4996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:42.165035 5005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 151818fb15694865b7b10fd0bc64c2b0. Latest consensus state: current_term: 1 leader_uuid: "151818fb15694865b7b10fd0bc64c2b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151818fb15694865b7b10fd0bc64c2b0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46641 } } }
I20260808 10:39:42.165076 5006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "151818fb15694865b7b10fd0bc64c2b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151818fb15694865b7b10fd0bc64c2b0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46641 } } }
I20260808 10:39:42.165171 5005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:42.165176 5006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:42.165621 5014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:42.167030 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:42.152132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.1.194.126:46641)
0808 10:39:42.152676 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:42.152698 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:42.152744 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:39:42.153614 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:42.153637 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:42.154028 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:42.154388 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:42.155142 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:42.156560 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:42.157420 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:42.158168 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:42.158330 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:42.160965 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:42.160969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:42.160981 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:42.161008 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:42.163025 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:42.163576 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:42.163581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:42.163587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:42.163733 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:42.164040 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:42.164045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:42.164047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:42.164583 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:42.165597 (+ 1014us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:42.165766 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:39:42.166092 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":134,"threads_started":1}
I20260808 10:39:42.168776 5014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:42.170234 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:42.174292 5014 catalog_manager.cc:1347] Generated new cluster ID: 736f48e735d74397858345a4e388e5db
I20260808 10:39:42.174365 5014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:42.194590 5014 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:42.194861 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:42.189063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.1.194.126:46641)
0808 10:39:42.189193 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:42.189198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:42.189524 (+ 326us) server_negotiation.cc:407] Connection header received
0808 10:39:42.189644 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:42.189647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:42.189706 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:42.189799 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:42.190363 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:42.191163 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:42.192340 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:42.192703 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:42.192828 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:42.193762 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:42.193765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:42.193768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:42.194368 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:42.194403 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:42.194423 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:42.194512 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:42.194664 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:39:42.194735 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:39:42.196097 5014 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:42.201050 5014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 151818fb15694865b7b10fd0bc64c2b0: Generated new TSK 0
I20260808 10:39:42.201906 5014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:42.208828 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 4918
2026-08-08T10:39:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1541 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:42 dist-test-slave-ss4n krb5kdc[5035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:42 dist-test-slave-ss4n krb5kdc[5035](info): set up 2 sockets
Aug 08 10:39:42 dist-test-slave-ss4n krb5kdc[5035](info): commencing operation
krb5kdc: starting...
W20260808 10:39:43.355499 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:43 dist-test-slave-ss4n krb5kdc[5035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:43.489931 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:33739
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:33739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:43.607533 5048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:43.607805 5048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:43.607848 5048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:43.611922 5048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:43.611981 5048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:43.612001 5048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:43.612018 5048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:43.612059 5048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:43.617928 5048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:33739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:43.619012 5048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:43.620062 5048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:43.629287 5053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:43.629352 5054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:43.629441 5048 server_base.cc:1034] running on GCE node
W20260808 10:39:43.629266 5056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:43.630138 5048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:43.631268 5048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:43.632457 5048 hybrid_clock.cc:648] HybridClock initialized: now 1786185583632427 us; error 44 us; skew 500 ppm
Aug 08 10:39:43 dist-test-slave-ss4n krb5kdc[5035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:43.635207 5048 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:43.636106 5048 webserver.cc:460] Webserver started at http://127.1.194.126:46249/ using document root <none> and password file <none>
I20260808 10:39:43.636708 5048 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:43.636798 5048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:43.637030 5048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:43.639098 5048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "0bbc295107784387a34a5ab1d0005654"
format_stamp: "Formatted at 2026-08-08 10:39:43 on dist-test-slave-ss4n"
server_key: "51247c102ff8bb43e553ba56a0d550b6"
server_key_iv: "0e7287a0cedfed9c1505aa5b0c113027"
server_key_version: "encryptionkey@0"
I20260808 10:39:43.639678 5048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "0bbc295107784387a34a5ab1d0005654"
format_stamp: "Formatted at 2026-08-08 10:39:43 on dist-test-slave-ss4n"
server_key: "51247c102ff8bb43e553ba56a0d550b6"
server_key_iv: "0e7287a0cedfed9c1505aa5b0c113027"
server_key_version: "encryptionkey@0"
I20260808 10:39:43.643422 5048 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:39:43.645843 5065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:43.647042 5048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:43.647188 5048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "0bbc295107784387a34a5ab1d0005654"
format_stamp: "Formatted at 2026-08-08 10:39:43 on dist-test-slave-ss4n"
server_key: "51247c102ff8bb43e553ba56a0d550b6"
server_key_iv: "0e7287a0cedfed9c1505aa5b0c113027"
server_key_version: "encryptionkey@0"
I20260808 10:39:43.647298 5048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:43.656441 5048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:43.659319 5048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:43.659519 5048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:43.666774 5048 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:33739
I20260808 10:39:43.667085 5124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:33739 every 8 connection(s)
I20260808 10:39:43.667732 5048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:43.670521 5125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:43.676739 5125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654: Bootstrap starting.
I20260808 10:39:43.677098 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5048
I20260808 10:39:43.677264 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:43.677601 1801 external_mini_cluster.cc:1428] Setting key 7b0e563a05d29169cf79907c8aff7a9c
I20260808 10:39:43.679760 5125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:43.680559 5125 log.cc:824] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:43.682650 5125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654: No bootstrap required, opened a new log
Aug 08 10:39:43 dist-test-slave-ss4n krb5kdc[5035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:43.687184 5125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbc295107784387a34a5ab1d0005654" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 33739 } }
I20260808 10:39:43.687528 5125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:43.687635 5125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbc295107784387a34a5ab1d0005654, State: Initialized, Role: FOLLOWER
I20260808 10:39:43.688236 5125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbc295107784387a34a5ab1d0005654" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 33739 } }
I20260808 10:39:43.688412 5125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:43.688529 5125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:43.688647 5125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:43.689790 5125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbc295107784387a34a5ab1d0005654" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 33739 } }
I20260808 10:39:43.690201 5125 leader_election.cc:302] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bbc295107784387a34a5ab1d0005654; no voters:
I20260808 10:39:43.690619 5125 leader_election.cc:288] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:43.690701 5130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:43.690919 5130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [term 1 LEADER]: Becoming Leader. State: Replica: 0bbc295107784387a34a5ab1d0005654, State: Running, Role: LEADER
I20260808 10:39:43.691257 5130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbc295107784387a34a5ab1d0005654" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 33739 } }
I20260808 10:39:43.692021 5125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:43.692934 5131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bbc295107784387a34a5ab1d0005654. Latest consensus state: current_term: 1 leader_uuid: "0bbc295107784387a34a5ab1d0005654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbc295107784387a34a5ab1d0005654" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 33739 } } }
I20260808 10:39:43.693063 5131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:43.693471 5141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:43.693444 5132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bbc295107784387a34a5ab1d0005654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbc295107784387a34a5ab1d0005654" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 33739 } } }
I20260808 10:39:43.693547 5132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:43.693689 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:43.679006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.1.194.126:33739)
0808 10:39:43.679515 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:43.679550 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:43.679596 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:39:43.680263 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:43.680282 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:43.680597 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:43.680968 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:43.681858 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:43.682895 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:43.683772 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:43.684236 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:43.684409 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:43.687531 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:43.687539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:43.687559 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:43.687593 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:43.689617 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:43.690184 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:43.690188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:43.690193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:43.690357 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:43.690683 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:43.690687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:43.690689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:43.691286 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:43.692067 (+ 781us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:43.692221 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 10:39:43.692542 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":151,"threads_started":1}
I20260808 10:39:43.696991 5141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:43.703035 5141 catalog_manager.cc:1347] Generated new cluster ID: 4b07b23a35304358b03764ef8c24ac36
I20260808 10:39:43.703125 5141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:43.713510 5141 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:43.714838 5141 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:43.720090 5141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bbc295107784387a34a5ab1d0005654: Generated new TSK 0
I20260808 10:39:43.721004 5141 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:43.760447 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:33739
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:43.874938 5155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:43.875207 5155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:43.875278 5155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:43.879561 5155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:43.879633 5155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:43.879741 5155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:43.885213 5155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:43.886356 5155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:43.887400 5155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:43.897275 5162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:43.897307 5165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:43.897413 5155 server_base.cc:1034] running on GCE node
W20260808 10:39:43.897356 5163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:43.897989 5155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:43.898693 5155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:43.899883 5155 hybrid_clock.cc:648] HybridClock initialized: now 1786185583899846 us; error 49 us; skew 500 ppm
Aug 08 10:39:43 dist-test-slave-ss4n krb5kdc[5035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:43.902954 5155 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:43.903859 5155 webserver.cc:460] Webserver started at http://127.1.194.65:44117/ using document root <none> and password file <none>
I20260808 10:39:43.904453 5155 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:43.904551 5155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:43.904788 5155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:43.907001 5155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "664ee41380f2415fba76928a3c9c0696"
format_stamp: "Formatted at 2026-08-08 10:39:43 on dist-test-slave-ss4n"
server_key: "b7b59143036dd905a7227f4ba6d79751"
server_key_iv: "51519e1a6f7b1348b7068646869872fe"
server_key_version: "encryptionkey@0"
I20260808 10:39:43.907611 5155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "664ee41380f2415fba76928a3c9c0696"
format_stamp: "Formatted at 2026-08-08 10:39:43 on dist-test-slave-ss4n"
server_key: "b7b59143036dd905a7227f4ba6d79751"
server_key_iv: "51519e1a6f7b1348b7068646869872fe"
server_key_version: "encryptionkey@0"
I20260808 10:39:43.911687 5155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:43.914294 5174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:43.916028 5155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:43.916163 5155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "664ee41380f2415fba76928a3c9c0696"
format_stamp: "Formatted at 2026-08-08 10:39:43 on dist-test-slave-ss4n"
server_key: "b7b59143036dd905a7227f4ba6d79751"
server_key_iv: "51519e1a6f7b1348b7068646869872fe"
server_key_version: "encryptionkey@0"
I20260808 10:39:43.916301 5155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:43.932696 5155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:43.935796 5155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:43.935950 5155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:43.936473 5155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:43.937310 5155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:43.937359 5155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:43.937399 5155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:43.937414 5155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:43.947098 5155 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:46863
I20260808 10:39:43.947117 5300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:46863 every 8 connection(s)
I20260808 10:39:43.948177 5155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:43.957970 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5155
I20260808 10:39:43.958143 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:43.958535 1801 external_mini_cluster.cc:1428] Setting key 9d9fbb692947f32f8d0855618cfdbd7b
Aug 08 10:39:43 dist-test-slave-ss4n krb5kdc[5035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:43.963754 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:43.950439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:48765 (local address 127.1.194.126:33739)
0808 10:39:43.950610 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:43.950615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:43.951639 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 10:39:43.953088 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:43.953091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:43.953150 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:43.953257 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:43.954524 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:43.955060 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:43.956208 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:43.956449 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:43.956566 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:43.959509 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:43.959513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:43.959516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:43.959542 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:43.961159 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:43.961886 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:43.961888 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:43.961889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:43.961967 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:43.962418 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:43.962423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:43.962428 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:43.962707 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:43.962821 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:43.963374 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 10:39:43.963537 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:39:43.964571 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:43.950966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:33739 (local address 127.1.194.65:48765)
0808 10:39:43.951475 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:43.951525 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:43.952871 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:43.953431 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:43.953438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:43.953889 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:43.954371 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:43.954384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:43.955197 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:43.955200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:43.956071 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:43.956080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:43.956608 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:43.956612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:43.956789 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:43.957398 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:43.957414 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:43.959314 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:43.961287 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:43.961297 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:43.961318 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:43.961783 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:43.962066 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:43.962073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:43.962078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:43.962300 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:43.962832 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:43.962838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:43.963109 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 10:39:43.963274 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:39:43.963570 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260808 10:39:43.965946 5301 heartbeater.cc:347] Connected to a master server at 127.1.194.126:33739
I20260808 10:39:43.966228 5301 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:43.966817 5301 heartbeater.cc:511] Master 127.1.194.126:33739 requested a full tablet report, sending...
I20260808 10:39:43.968400 5085 ts_manager.cc:194] Registered new tserver with Master: 664ee41380f2415fba76928a3c9c0696 (127.1.194.65:46863)
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:43.969794 5085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:48765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:43.988917 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:33739
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:44.101625 5307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:44.101922 5307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:44.101966 5307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:44.106060 5307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:44.106123 5307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:44.106189 5307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:44.111538 5307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:44.112661 5307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:44.113689 5307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:44.123520 5312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:44.123533 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:44.123637 5307 server_base.cc:1034] running on GCE node
W20260808 10:39:44.123533 5313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:44.124207 5307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:44.124871 5307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:44.126065 5307 hybrid_clock.cc:648] HybridClock initialized: now 1786185584126044 us; error 38 us; skew 500 ppm
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:44.128973 5307 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:44.129992 5307 webserver.cc:460] Webserver started at http://127.1.194.66:33043/ using document root <none> and password file <none>
I20260808 10:39:44.130560 5307 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:44.130663 5307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:44.130895 5307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:44.132956 5307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "cb2d184d49d6432dbe77002887f257c3"
format_stamp: "Formatted at 2026-08-08 10:39:44 on dist-test-slave-ss4n"
server_key: "7e54e19cf342322993299bf17b57450d"
server_key_iv: "c79735c71e7124bd57d9402e01bfa4ca"
server_key_version: "encryptionkey@0"
I20260808 10:39:44.133620 5307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "cb2d184d49d6432dbe77002887f257c3"
format_stamp: "Formatted at 2026-08-08 10:39:44 on dist-test-slave-ss4n"
server_key: "7e54e19cf342322993299bf17b57450d"
server_key_iv: "c79735c71e7124bd57d9402e01bfa4ca"
server_key_version: "encryptionkey@0"
I20260808 10:39:44.137275 5307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:44.139624 5322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:44.140818 5307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:39:44.140921 5307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "cb2d184d49d6432dbe77002887f257c3"
format_stamp: "Formatted at 2026-08-08 10:39:44 on dist-test-slave-ss4n"
server_key: "7e54e19cf342322993299bf17b57450d"
server_key_iv: "c79735c71e7124bd57d9402e01bfa4ca"
server_key_version: "encryptionkey@0"
I20260808 10:39:44.141004 5307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:44.156880 5307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:44.159891 5307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:44.160043 5307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:44.160547 5307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:44.161305 5307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:44.161350 5307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:44.161389 5307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:44.161404 5307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:44.171473 5307 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:39093
I20260808 10:39:44.171622 5444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:39093 every 8 connection(s)
I20260808 10:39:44.172461 5307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:44.176194 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5307
I20260808 10:39:44.176334 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:44.176620 1801 external_mini_cluster.cc:1428] Setting key 547ecbb6d9681803b903b1db517d6f27
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:44.187531 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:44.174490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:45309 (local address 127.1.194.126:33739)
0808 10:39:44.174658 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:44.174662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:44.175440 (+ 778us) server_negotiation.cc:407] Connection header received
0808 10:39:44.177015 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:44.177019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:44.177076 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:44.177197 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:44.178558 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.179097 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.180218 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.180548 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.180662 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:44.183415 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:44.183421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:44.183427 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:44.183461 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:44.185312 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:44.185901 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:44.185906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:44.185910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:44.186006 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:44.186358 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:44.186361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:44.186363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:44.186612 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:44.186724 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:44.187213 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 10:39:44.187356 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:44.188323 5447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:44.174855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:33739 (local address 127.1.194.66:45309)
0808 10:39:44.175283 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:44.175324 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:44.176804 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:44.177384 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:44.177392 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:44.177904 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:44.178418 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:44.178432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.179225 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.179229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:44.180087 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:44.180096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.180704 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.180708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:44.180874 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:44.181543 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:44.181566 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:44.183201 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:44.185443 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:44.185448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:44.185459 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:44.185776 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:44.186110 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:44.186112 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:44.186113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:44.186245 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:44.186717 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:44.186722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:44.186945 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 10:39:44.187093 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 10:39:44.187391 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260808 10:39:44.189672 5445 heartbeater.cc:347] Connected to a master server at 127.1.194.126:33739
I20260808 10:39:44.189903 5445 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:44.190454 5445 heartbeater.cc:511] Master 127.1.194.126:33739 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:44.191704 5085 ts_manager.cc:194] Registered new tserver with Master: cb2d184d49d6432dbe77002887f257c3 (127.1.194.66:39093)
I20260808 10:39:44.192600 5085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:45309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:44.210198 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:33739
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:44.322544 5451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:44.322847 5451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:44.322923 5451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:44.326957 5451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:44.327064 5451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:44.327168 5451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:44.332559 5451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:44.333763 5451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:44.334867 5451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:44.344878 5459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:44.345098 5451 server_base.cc:1034] running on GCE node
W20260808 10:39:44.344899 5457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:44.344882 5456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:44.345664 5451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:44.346383 5451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:44.347569 5451 hybrid_clock.cc:648] HybridClock initialized: now 1786185584347557 us; error 28 us; skew 500 ppm
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:44.350457 5451 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:44.351418 5451 webserver.cc:460] Webserver started at http://127.1.194.67:33957/ using document root <none> and password file <none>
I20260808 10:39:44.352007 5451 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:44.352104 5451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:44.352346 5451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:44.354477 5451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "273b795953c14a279089c3a0bf0b4921"
format_stamp: "Formatted at 2026-08-08 10:39:44 on dist-test-slave-ss4n"
server_key: "09ad5512981366f5427c9b7fe523fb7a"
server_key_iv: "74fbbd44a97c4131bf42081f56f6ad35"
server_key_version: "encryptionkey@0"
I20260808 10:39:44.355063 5451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "273b795953c14a279089c3a0bf0b4921"
format_stamp: "Formatted at 2026-08-08 10:39:44 on dist-test-slave-ss4n"
server_key: "09ad5512981366f5427c9b7fe523fb7a"
server_key_iv: "74fbbd44a97c4131bf42081f56f6ad35"
server_key_version: "encryptionkey@0"
I20260808 10:39:44.358892 5451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:44.361438 5468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:44.362815 5451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:44.362963 5451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "273b795953c14a279089c3a0bf0b4921"
format_stamp: "Formatted at 2026-08-08 10:39:44 on dist-test-slave-ss4n"
server_key: "09ad5512981366f5427c9b7fe523fb7a"
server_key_iv: "74fbbd44a97c4131bf42081f56f6ad35"
server_key_version: "encryptionkey@0"
I20260808 10:39:44.363085 5451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:44.379679 5451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:44.383139 5451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:44.383376 5451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:44.383996 5451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:44.384850 5451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:44.384922 5451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:44.384992 5451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:44.385044 5451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:44.394685 5451 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:40619
I20260808 10:39:44.394709 5595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:40619 every 8 connection(s)
I20260808 10:39:44.395617 5451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:44.397482 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5451
I20260808 10:39:44.397639 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:44.397953 1801 external_mini_cluster.cc:1428] Setting key 23877f38b2394cdf6856b155cf09d150
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:44.409646 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:44.397641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:47891 (local address 127.1.194.126:33739)
0808 10:39:44.397795 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:44.397800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:44.398804 (+ 1004us) server_negotiation.cc:407] Connection header received
0808 10:39:44.400425 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:44.400429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:44.400486 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:44.400591 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:44.401876 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.402420 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.403543 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.403768 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.403880 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:44.406420 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:44.406423 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:44.406425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:44.406450 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:44.407779 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:44.408269 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:44.408272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:44.408273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:44.408347 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:44.408650 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:44.408652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:44.408653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:44.408820 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:44.408894 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:44.409357 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 10:39:44.409456 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:39:44.410557 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:44.398074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:33739 (local address 127.1.194.67:47891)
0808 10:39:44.398643 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:44.398706 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:44.400213 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:44.400771 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:44.400798 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:44.401161 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:44.401730 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:44.401746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.402552 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.402556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:44.403408 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:44.403417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.403917 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.403921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:44.404084 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:44.404680 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:44.404696 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:44.406266 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:44.407885 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:44.407889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:44.407901 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:44.408171 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:44.408427 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:44.408429 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:44.408430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:44.408560 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:44.408904 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:44.408909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:44.409130 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 10:39:44.409292 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 10:39:44.409619 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":158,"threads_started":1}
I20260808 10:39:44.411643 5596 heartbeater.cc:347] Connected to a master server at 127.1.194.126:33739
I20260808 10:39:44.411918 5596 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:44.412460 5596 heartbeater.cc:511] Master 127.1.194.126:33739 requested a full tablet report, sending...
I20260808 10:39:44.413455 5085 ts_manager.cc:194] Registered new tserver with Master: 273b795953c14a279089c3a0bf0b4921 (127.1.194.67:40619)
I20260808 10:39:44.414171 5085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:47891
I20260808 10:39:44.424121 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185584, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185584, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:44.451951 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:44.442946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.1.194.126:33739)
0808 10:39:44.443127 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:44.443133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:44.443283 (+ 150us) server_negotiation.cc:407] Connection header received
0808 10:39:44.443476 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:44.443479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:44.443525 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:44.443615 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:44.444356 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.444888 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.445826 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:44.446047 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:44.446157 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:44.448694 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:44.448698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:44.448700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:44.448726 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:44.450143 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:44.450612 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:44.450615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:44.450617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:44.450754 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:44.451026 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:44.451031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:44.451034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:44.451276 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:44.451382 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:44.451597 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:39:44.451717 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 10:39:44.452421 5085 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51044
I20260808 10:39:44.453385 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5155
I20260808 10:39:44.462102 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5307
I20260808 10:39:44.469244 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5451
I20260808 10:39:44.475136 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5048
2026-08-08T10:39:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2267 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5611](info): set up 2 sockets
Aug 08 10:39:44 dist-test-slave-ss4n krb5kdc[5611](info): commencing operation
krb5kdc: starting...
W20260808 10:39:45.600205 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:45 dist-test-slave-ss4n krb5kdc[5611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:45.729346 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34323
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:45.843261 5625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:45.843571 5625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:45.843664 5625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:45.847826 5625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:45.847918 5625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:45.847954 5625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:45.847994 5625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:45.848032 5625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:45.853387 5625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:45.854508 5625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:45.855587 5625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:45.864037 5631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:45.864080 5633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:45.864051 5630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:45.864212 5625 server_base.cc:1034] running on GCE node
I20260808 10:39:45.864889 5625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:45.866037 5625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:45.867221 5625 hybrid_clock.cc:648] HybridClock initialized: now 1786185585867203 us; error 28 us; skew 500 ppm
Aug 08 10:39:45 dist-test-slave-ss4n krb5kdc[5611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185585, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:45.875089 5625 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:45.876150 5625 webserver.cc:460] Webserver started at http://127.1.194.126:34003/ using document root <none> and password file <none>
I20260808 10:39:45.876780 5625 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:45.876883 5625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:45.877153 5625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:45.879560 5625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "29310e97b5a24eb792fb9e8ee1da6ac2"
format_stamp: "Formatted at 2026-08-08 10:39:45 on dist-test-slave-ss4n"
server_key: "62b2837a3bad3d328db5111f3b3303c9"
server_key_iv: "9d07f106900b0399d68e74ebc025a306"
server_key_version: "encryptionkey@0"
I20260808 10:39:45.880162 5625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "29310e97b5a24eb792fb9e8ee1da6ac2"
format_stamp: "Formatted at 2026-08-08 10:39:45 on dist-test-slave-ss4n"
server_key: "62b2837a3bad3d328db5111f3b3303c9"
server_key_iv: "9d07f106900b0399d68e74ebc025a306"
server_key_version: "encryptionkey@0"
I20260808 10:39:45.884271 5625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:45.886970 5640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:45.888263 5625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:45.888418 5625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "29310e97b5a24eb792fb9e8ee1da6ac2"
format_stamp: "Formatted at 2026-08-08 10:39:45 on dist-test-slave-ss4n"
server_key: "62b2837a3bad3d328db5111f3b3303c9"
server_key_iv: "9d07f106900b0399d68e74ebc025a306"
server_key_version: "encryptionkey@0"
I20260808 10:39:45.888566 5625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:45.907982 5625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:45.911772 5625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:45.912034 5625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:45.921808 5625 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34323
I20260808 10:39:45.923249 5625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:45.924885 5699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34323 every 8 connection(s)
I20260808 10:39:45.926479 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5625
I20260808 10:39:45.926643 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:45.926951 1801 external_mini_cluster.cc:1428] Setting key 4898a95011871718a79f3b35111929e3
I20260808 10:39:45.927392 5700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:45.936148 5700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2: Bootstrap starting.
Aug 08 10:39:45 dist-test-slave-ss4n krb5kdc[5611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:45.938747 5700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:45.939749 5700 log.cc:824] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:45.941871 5700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2: No bootstrap required, opened a new log
I20260808 10:39:45.943451 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:45.928471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47276 (local address 127.1.194.126:34323)
0808 10:39:45.929024 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:45.929044 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:45.929091 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:39:45.930095 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:45.930125 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:45.930712 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:45.931186 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:45.932119 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:45.933454 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:45.934700 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:45.935179 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:45.935306 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:45.938301 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:45.938306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:45.938319 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:45.938346 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:45.940410 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:45.940887 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:45.940893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:45.940899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:45.941038 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:45.941337 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:45.941342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:45.941344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:45.941811 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:45.942037 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:45.942332 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 10:39:45.942647 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":176,"threads_started":1}
I20260808 10:39:45.945865 5700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34323 } }
I20260808 10:39:45.946138 5700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:45.946209 5700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29310e97b5a24eb792fb9e8ee1da6ac2, State: Initialized, Role: FOLLOWER
I20260808 10:39:45.946657 5700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34323 } }
I20260808 10:39:45.946805 5700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:45.946869 5700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:45.946960 5700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:45.947913 5700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34323 } }
I20260808 10:39:45.948261 5700 leader_election.cc:302] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29310e97b5a24eb792fb9e8ee1da6ac2; no voters:
I20260808 10:39:45.948590 5700 leader_election.cc:288] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:45.948764 5707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:45.949059 5707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 29310e97b5a24eb792fb9e8ee1da6ac2, State: Running, Role: LEADER
I20260808 10:39:45.949533 5707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34323 } }
I20260808 10:39:45.949926 5700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:45.951460 5709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29310e97b5a24eb792fb9e8ee1da6ac2. Latest consensus state: current_term: 1 leader_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34323 } } }
I20260808 10:39:45.951637 5709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:45.951968 5708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29310e97b5a24eb792fb9e8ee1da6ac2" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34323 } } }
I20260808 10:39:45.952893 5708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2 [sys.catalog]: This master's current role is: LEADER
W20260808 10:39:45.955525 5723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:45.955606 5723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:45.955704 5722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:45.956557 5722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:45.963018 5722 catalog_manager.cc:1347] Generated new cluster ID: ef386da141594a67aff57d483abe3b35
I20260808 10:39:45.963133 5722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:45.969596 5722 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:45.970563 5722 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:45.983944 5722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29310e97b5a24eb792fb9e8ee1da6ac2: Generated new TSK 0
I20260808 10:39:45.984908 5722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:46.024998 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34323
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:46.143942 5730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:46.144196 5730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:46.144237 5730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:46.148294 5730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:46.148355 5730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:46.148411 5730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:46.154043 5730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:46.155069 5730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:46.156077 5730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:46.165732 5736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:46.165722 5738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:46.165710 5730 server_base.cc:1034] running on GCE node
W20260808 10:39:46.165688 5735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:46.166297 5730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:46.166864 5730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:46.168026 5730 hybrid_clock.cc:648] HybridClock initialized: now 1786185586168006 us; error 34 us; skew 500 ppm
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:46.170897 5730 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:46.171712 5730 webserver.cc:460] Webserver started at http://127.1.194.65:36613/ using document root <none> and password file <none>
I20260808 10:39:46.172225 5730 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:46.172282 5730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:46.172442 5730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:46.174508 5730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "170b9821c0e94ae6a8c80224143b6af7"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "834d230baf7501d7c0211c2d12cdc912"
server_key_iv: "a0075e80e896cc1d7dc2a7f4aafd4613"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.175024 5730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "170b9821c0e94ae6a8c80224143b6af7"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "834d230baf7501d7c0211c2d12cdc912"
server_key_iv: "a0075e80e896cc1d7dc2a7f4aafd4613"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.178596 5730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 10:39:46.180910 5746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.182229 5730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:46.182385 5730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "170b9821c0e94ae6a8c80224143b6af7"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "834d230baf7501d7c0211c2d12cdc912"
server_key_iv: "a0075e80e896cc1d7dc2a7f4aafd4613"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.182585 5730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:46.197171 5730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:46.200464 5730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:46.200685 5730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:46.201289 5730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:46.202211 5730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:46.202286 5730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.202356 5730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:46.202407 5730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.211318 5730 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:44235
I20260808 10:39:46.211381 5875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:44235 every 8 connection(s)
I20260808 10:39:46.212260 5730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:46.212806 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5730
I20260808 10:39:46.212945 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:46.213327 1801 external_mini_cluster.cc:1428] Setting key a9670921855f2bfdea0b360738e7e338
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:46.226949 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.214251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:54207 (local address 127.1.194.126:34323)
0808 10:39:46.214390 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:46.214394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:46.215409 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 10:39:46.216770 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:46.216773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:46.216829 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:46.216911 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:46.218116 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.218659 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.219782 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.220023 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.220129 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.223005 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:46.223008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:46.223010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:46.223034 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:46.224726 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.225245 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.225249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.225251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.225337 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.225945 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.225947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.225948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.226126 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:46.226215 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:46.226724 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 10:39:46.226827 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:39:46.228206 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.214781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34323 (local address 127.1.194.65:54207)
0808 10:39:46.215262 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:46.215304 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:46.216564 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:46.217079 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:46.217085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:46.217435 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:46.217970 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:46.217983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.218792 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.218800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:46.219654 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:46.219664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.220189 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.220194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:46.220358 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.221033 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:46.221048 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:46.222834 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:46.224849 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:46.224854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:46.224866 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:46.225143 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:46.225702 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:46.225705 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:46.225707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:46.225846 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:46.226238 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:46.226243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:46.226505 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 10:39:46.226671 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:39:46.227113 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":158,"threads_started":1}
I20260808 10:39:46.229540 5876 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34323
I20260808 10:39:46.229811 5876 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:46.230491 5876 heartbeater.cc:511] Master 127.1.194.126:34323 requested a full tablet report, sending...
I20260808 10:39:46.232173 5654 ts_manager.cc:194] Registered new tserver with Master: 170b9821c0e94ae6a8c80224143b6af7 (127.1.194.65:44235)
I20260808 10:39:46.233335 5654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:54207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:46.245819 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34323
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:46.359083 5882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:46.359413 5882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:46.359494 5882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:46.363731 5882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:46.363828 5882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:46.363926 5882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:46.369376 5882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:46.370525 5882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:46.371578 5882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:46.381467 5888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:46.381503 5890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:46.381503 5887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:46.381753 5882 server_base.cc:1034] running on GCE node
I20260808 10:39:46.382301 5882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:46.383015 5882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:46.384203 5882 hybrid_clock.cc:648] HybridClock initialized: now 1786185586384175 us; error 51 us; skew 500 ppm
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:46.387422 5882 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:46.388372 5882 webserver.cc:460] Webserver started at http://127.1.194.66:35525/ using document root <none> and password file <none>
I20260808 10:39:46.388945 5882 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:46.389037 5882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:46.389276 5882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:46.391400 5882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "1ce9e76b8885461bafe594bf3adc26ea"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "5d05484574f7de266cb5c542d5c3832c"
server_key_iv: "9278d366ca5dea3b5e61d455a07f2c3e"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.391988 5882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "1ce9e76b8885461bafe594bf3adc26ea"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "5d05484574f7de266cb5c542d5c3832c"
server_key_iv: "9278d366ca5dea3b5e61d455a07f2c3e"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.395795 5882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:46.398268 5898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.399461 5882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:46.399597 5882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "1ce9e76b8885461bafe594bf3adc26ea"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "5d05484574f7de266cb5c542d5c3832c"
server_key_iv: "9278d366ca5dea3b5e61d455a07f2c3e"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.399721 5882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:46.415647 5882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:46.418736 5882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:46.418962 5882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:46.419615 5882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:46.420481 5882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:46.420557 5882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.420643 5882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:46.420693 5882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.430133 5882 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:40967
I20260808 10:39:46.430179 6022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:40967 every 8 connection(s)
I20260808 10:39:46.431077 5882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:46.433403 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 5882
I20260808 10:39:46.433552 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:46.433888 1801 external_mini_cluster.cc:1428] Setting key 772f626f5eddf40c469fef68ffe9a906
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:46.447055 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.433112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:56403 (local address 127.1.194.126:34323)
0808 10:39:46.433295 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:46.433299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:46.434381 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 10:39:46.435968 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:46.435974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:46.436033 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:46.436127 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:46.437563 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.438100 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.439420 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.439923 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.440059 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.443208 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:46.443211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:46.443215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:46.443253 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:46.444984 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.445566 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.445574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.445575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.445658 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.446055 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.446058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.446059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.446233 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:46.446312 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:46.446797 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 10:39:46.446902 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 10:39:46.447963 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.433689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34323 (local address 127.1.194.66:56403)
0808 10:39:46.434205 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:46.434248 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:46.435758 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:46.436312 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:46.436319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:46.436735 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:46.437341 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:46.437367 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.438236 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.438242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:46.439279 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:46.439289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.440128 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.440135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:46.440293 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.441022 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:46.441038 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:46.443027 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:46.445105 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:46.445109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:46.445122 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:46.445382 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:46.445748 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:46.445751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:46.445753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:46.445957 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:46.446332 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:46.446337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:46.446571 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:39:46.446736 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:39:46.447025 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":147,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:46.449333 6023 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34323
I20260808 10:39:46.449699 6023 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:46.450377 6023 heartbeater.cc:511] Master 127.1.194.126:34323 requested a full tablet report, sending...
I20260808 10:39:46.451581 5654 ts_manager.cc:194] Registered new tserver with Master: 1ce9e76b8885461bafe594bf3adc26ea (127.1.194.66:40967)
I20260808 10:39:46.452298 5654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:56403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:46.469599 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34323
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:46.584172 6028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:46.584465 6028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:46.584542 6028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:46.588814 6028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:46.588899 6028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:46.588991 6028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:46.594201 6028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:46.595268 6028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:46.596342 6028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:46.606244 6039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:46.606267 6037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:46.606244 6036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:46.606282 6028 server_base.cc:1034] running on GCE node
I20260808 10:39:46.606863 6028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:46.607487 6028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:46.608675 6028 hybrid_clock.cc:648] HybridClock initialized: now 1786185586608665 us; error 30 us; skew 500 ppm
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:46.611428 6028 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:46.612323 6028 webserver.cc:460] Webserver started at http://127.1.194.67:39333/ using document root <none> and password file <none>
I20260808 10:39:46.612825 6028 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:46.612890 6028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:46.613049 6028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:46.615124 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "0d2701bce3e5443b9c3cf92f903fd583"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "6289ac1526fae47aa7fd8d168867f827"
server_key_iv: "3b7cf198ff6c08d46c834d7c1ec9404a"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.615626 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "0d2701bce3e5443b9c3cf92f903fd583"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "6289ac1526fae47aa7fd8d168867f827"
server_key_iv: "3b7cf198ff6c08d46c834d7c1ec9404a"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.619302 6028 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:39:46.621685 6047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.622967 6028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:46.623065 6028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "0d2701bce3e5443b9c3cf92f903fd583"
format_stamp: "Formatted at 2026-08-08 10:39:46 on dist-test-slave-ss4n"
server_key: "6289ac1526fae47aa7fd8d168867f827"
server_key_iv: "3b7cf198ff6c08d46c834d7c1ec9404a"
server_key_version: "encryptionkey@0"
I20260808 10:39:46.623180 6028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:46.636906 6028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:46.639688 6028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:46.639834 6028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:46.640341 6028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:46.641088 6028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:46.641132 6028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.641171 6028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:46.641194 6028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:46.650087 6028 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:40115
I20260808 10:39:46.650105 6167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:40115 every 8 connection(s)
I20260808 10:39:46.651122 6028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:46.656867 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6028
I20260808 10:39:46.656971 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:46.657270 1801 external_mini_cluster.cc:1428] Setting key 48a3863f0cd0ce508dd7a73ca24dd20d
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:46.664929 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.652969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:32935 (local address 127.1.194.126:34323)
0808 10:39:46.653145 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:46.653150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:46.654028 (+ 878us) server_negotiation.cc:407] Connection header received
0808 10:39:46.655374 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:46.655377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:46.655425 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:46.655502 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:46.656679 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.657323 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.658856 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.659090 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.659207 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.661796 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:46.661799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:46.661801 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:46.661824 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:46.663059 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.663537 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.663540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.663541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.663621 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.663933 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.663935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.663936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.664103 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:46.664178 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:46.664682 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 10:39:46.664812 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:39:46.665750 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.653382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34323 (local address 127.1.194.67:32935)
0808 10:39:46.653865 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:46.653905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:46.655183 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:46.655682 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:46.655688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:46.656024 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:46.656493 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:46.656505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.657461 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.657464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:46.658731 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:46.658741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.659247 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.659251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:46.659431 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.660060 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:46.660076 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:46.661638 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:46.663168 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:46.663173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:46.663184 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:46.663436 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:46.663715 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:46.663718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:46.663719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:46.663848 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:46.664196 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:46.664202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:46.664425 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 10:39:46.664588 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 10:39:46.664835 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":131,"threads_started":1}
I20260808 10:39:46.666867 6168 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34323
I20260808 10:39:46.667114 6168 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:46.667611 6168 heartbeater.cc:511] Master 127.1.194.126:34323 requested a full tablet report, sending...
I20260808 10:39:46.668664 5654 ts_manager.cc:194] Registered new tserver with Master: 0d2701bce3e5443b9c3cf92f903fd583 (127.1.194.67:40115)
I20260808 10:39:46.669255 5654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:32935
I20260808 10:39:46.671352 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:46.698769 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.688948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.1.194.65:44235)
0808 10:39:46.689269 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:46.689287 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:46.689317 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:39:46.689383 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:46.689397 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:46.689667 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:46.689826 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:46.690543 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.691324 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.692191 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.692579 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.692692 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.695390 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:46.695399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:46.695412 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:46.695435 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:46.696913 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.697360 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.697366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.697367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.697442 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.697771 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.697775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.697776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.698030 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:46.698264 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:46.698431 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:39:46.698581 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":92,"threads_started":1}
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[5611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:46.710582 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:46.701670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.1.194.126:34323)
0808 10:39:46.701824 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:46.701829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:46.701844 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:39:46.701946 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:46.701949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:46.701999 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:46.702111 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:46.702775 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.703355 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.704243 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:46.704465 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:46.704576 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:46.707138 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:46.707141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:46.707143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:46.707169 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:46.708718 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.709150 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.709153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.709154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.709232 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:46.709642 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:46.709645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:46.709647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:46.709887 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:46.710008 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:46.710260 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 10:39:46.710363 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 10:39:46.711017 5654 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47280
I20260808 10:39:46.711683 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5730
I20260808 10:39:46.718914 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5882
I20260808 10:39:46.725001 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6028
I20260808 10:39:46.732308 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 5625
2026-08-08T10:39:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2257 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[6181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[6181](info): set up 2 sockets
Aug 08 10:39:46 dist-test-slave-ss4n krb5kdc[6181](info): commencing operation
krb5kdc: starting...
W20260808 10:39:47.911906 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:48.049471 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:44487
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:44487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:48.161211 6196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:48.161562 6196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:48.161653 6196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:48.166038 6196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:48.166131 6196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:48.166170 6196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:48.166208 6196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:48.166244 6196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:48.171518 6196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:44487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:44487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:48.172667 6196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:48.173765 6196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:48.182152 6205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:48.182204 6203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:48.182152 6202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:48.182452 6196 server_base.cc:1034] running on GCE node
I20260808 10:39:48.183065 6196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:48.184119 6196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:48.185300 6196 hybrid_clock.cc:648] HybridClock initialized: now 1786185588185283 us; error 31 us; skew 500 ppm
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:48.188256 6196 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:48.189071 6196 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:48.189169 6196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:48.189386 6196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:48.191532 6196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "b61f87fdf6b64376acc077cc0305e452"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "e8101275caec24b529a501f6084d610b"
server_key_iv: "fc1fad1c7b254268141f93112e1052aa"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.192106 6196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "b61f87fdf6b64376acc077cc0305e452"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "e8101275caec24b529a501f6084d610b"
server_key_iv: "fc1fad1c7b254268141f93112e1052aa"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.195847 6196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:39:48.198770 6211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.200006 6196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:48.200155 6196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "b61f87fdf6b64376acc077cc0305e452"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "e8101275caec24b529a501f6084d610b"
server_key_iv: "fc1fad1c7b254268141f93112e1052aa"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.200284 6196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:48.233620 6196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:48.237124 6196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:48.237344 6196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:48.244792 6196 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:44487
I20260808 10:39:48.245108 6269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:44487 every 8 connection(s)
I20260808 10:39:48.245564 6196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:48.247200 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6196
I20260808 10:39:48.247365 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:48.247689 1801 external_mini_cluster.cc:1428] Setting key c23a385fe0c60e9f038f2bdc22674b21
I20260808 10:39:48.248706 6270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:48.256280 6270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452: Bootstrap starting.
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:48.259312 6270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:48.260215 6270 log.cc:824] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:48.262657 6270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452: No bootstrap required, opened a new log
I20260808 10:39:48.263855 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.249198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.1.194.126:44487)
0808 10:39:48.249678 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:48.249709 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:48.249747 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:39:48.250428 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:48.250444 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:48.250770 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:48.251064 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:48.252140 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.253271 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.254652 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.255158 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.255286 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.258624 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:48.258629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:48.258645 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:48.258675 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:48.260773 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.261332 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.261337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.261342 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.261453 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.261799 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.261801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.261802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.262226 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:48.262367 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:48.262783 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 10:39:48.263147 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":166,"threads_started":1}
I20260808 10:39:48.266242 6270 raft_consensus.cc:348] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61f87fdf6b64376acc077cc0305e452" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44487 } }
I20260808 10:39:48.266458 6270 raft_consensus.cc:374] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:48.266535 6270 raft_consensus.cc:729] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61f87fdf6b64376acc077cc0305e452, State: Initialized, Role: FOLLOWER
I20260808 10:39:48.266960 6270 consensus_queue.cc:260] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61f87fdf6b64376acc077cc0305e452" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44487 } }
I20260808 10:39:48.267112 6270 raft_consensus.cc:388] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:48.267174 6270 raft_consensus.cc:482] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:48.267288 6270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:48.268218 6270 raft_consensus.cc:504] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61f87fdf6b64376acc077cc0305e452" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44487 } }
I20260808 10:39:48.268548 6270 leader_election.cc:302] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b61f87fdf6b64376acc077cc0305e452; no voters:
I20260808 10:39:48.268850 6270 leader_election.cc:288] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:48.269026 6275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:48.269275 6275 raft_consensus.cc:686] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [term 1 LEADER]: Becoming Leader. State: Replica: b61f87fdf6b64376acc077cc0305e452, State: Running, Role: LEADER
I20260808 10:39:48.269631 6275 consensus_queue.cc:237] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61f87fdf6b64376acc077cc0305e452" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44487 } }
I20260808 10:39:48.270126 6270 sys_catalog.cc:564] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:48.271258 6277 sys_catalog.cc:455] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b61f87fdf6b64376acc077cc0305e452. Latest consensus state: current_term: 1 leader_uuid: "b61f87fdf6b64376acc077cc0305e452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61f87fdf6b64376acc077cc0305e452" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44487 } } }
I20260808 10:39:48.271406 6277 sys_catalog.cc:458] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:48.271790 6284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:48.271771 6276 sys_catalog.cc:455] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b61f87fdf6b64376acc077cc0305e452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61f87fdf6b64376acc077cc0305e452" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44487 } } }
I20260808 10:39:48.271942 6276 sys_catalog.cc:458] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:48.275242 6284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:48.281147 6284 catalog_manager.cc:1347] Generated new cluster ID: ba65a9fedced4223b3c3bc4d0ae26b43
I20260808 10:39:48.281222 6284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:48.288322 6284 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:48.289556 6284 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:48.302348 6284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452: Generated new TSK 0
I20260808 10:39:48.303179 6284 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:48.341163 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44487
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:48.456180 6297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:48.456465 6297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:48.456512 6297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:48.460731 6297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:48.460819 6297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:48.460881 6297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:48.466316 6297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:48.467427 6297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:48.468497 6297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:48.478276 6302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:48.478289 6303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:48.478392 6297 server_base.cc:1034] running on GCE node
W20260808 10:39:48.478286 6305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:48.478926 6297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:48.479586 6297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:48.480743 6297 hybrid_clock.cc:648] HybridClock initialized: now 1786185588480730 us; error 29 us; skew 500 ppm
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:48.483675 6297 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:48.484393 6297 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:48.484489 6297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:48.484722 6297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:48.486850 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "e5c44d8748504e92ae58a8a54127a00a"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "3c2bf7c87ed04100bdfc115eeb907d2e"
server_key_iv: "8fec801ff82d9578410c01f28e6ef6af"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.487412 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "e5c44d8748504e92ae58a8a54127a00a"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "3c2bf7c87ed04100bdfc115eeb907d2e"
server_key_iv: "8fec801ff82d9578410c01f28e6ef6af"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.491074 6297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:39:48.493436 6311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.494684 6297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:48.494815 6297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "e5c44d8748504e92ae58a8a54127a00a"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "3c2bf7c87ed04100bdfc115eeb907d2e"
server_key_iv: "8fec801ff82d9578410c01f28e6ef6af"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.494922 6297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:48.507974 6297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:48.511296 6297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:48.511498 6297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:48.512099 6297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:48.512905 6297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:48.512976 6297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.513051 6297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:48.513095 6297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.523366 6297 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:46633
I20260808 10:39:48.523396 6435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:46633 every 8 connection(s)
I20260808 10:39:48.523957 6297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:48.528852 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6297
I20260808 10:39:48.528997 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:48.529331 1801 external_mini_cluster.cc:1428] Setting key 1601dde254fa6b2a97d63b74c1ba5704
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:48.539669 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.525956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:47525 (local address 127.1.194.126:44487)
0808 10:39:48.526132 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:48.526136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:48.527064 (+ 928us) server_negotiation.cc:407] Connection header received
0808 10:39:48.528650 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:48.528654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:48.528726 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:48.528829 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:48.530369 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.531077 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.532306 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.532569 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.532687 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.535329 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:48.535333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:48.535335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:48.535359 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:48.537055 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.537729 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.537735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.537737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.537831 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.538260 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.538263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.538265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.538468 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:48.538595 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:48.539203 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 10:39:48.539400 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:39:48.541244 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.526323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44487 (local address 127.1.194.65:47525)
0808 10:39:48.526887 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:48.526932 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:48.528375 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:48.529064 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:48.529091 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:48.529603 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:48.530216 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:48.530232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.531212 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.531221 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:48.532169 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:48.532178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.532728 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.532733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:48.532909 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.533592 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:48.533609 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:48.535138 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:48.537204 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:48.537211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:48.537226 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:48.537614 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:48.537981 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:48.537984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:48.537987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:48.538144 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:48.538592 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:48.538600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:48.538856 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 10:39:48.539875 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0808 10:39:48.540430 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":177,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:39:48.542810 6436 heartbeater.cc:347] Connected to a master server at 127.1.194.126:44487
I20260808 10:39:48.543092 6436 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:48.543560 6436 heartbeater.cc:511] Master 127.1.194.126:44487 requested a full tablet report, sending...
I20260808 10:39:48.545053 6225 ts_manager.cc:194] Registered new tserver with Master: e5c44d8748504e92ae58a8a54127a00a (127.1.194.65:46633)
I20260808 10:39:48.546228 6225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:47525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:48.562486 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44487
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:48.678438 6441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:48.678695 6441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:48.678740 6441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:48.682955 6441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:48.683017 6441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:48.683099 6441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:48.688544 6441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:48.689790 6441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:48.690807 6441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:48.700491 6450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:48.700491 6447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:48.700572 6448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:48.700726 6441 server_base.cc:1034] running on GCE node
I20260808 10:39:48.701145 6441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:48.701872 6441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:48.703045 6441 hybrid_clock.cc:648] HybridClock initialized: now 1786185588703032 us; error 23 us; skew 500 ppm
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:48.706225 6441 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:48.706920 6441 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:48.706983 6441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:48.707154 6441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:48.709151 6441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "8ca76d6564404ff78136d1af7d2a8e3f"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "50f902c3739f75798d8d2021b917b394"
server_key_iv: "d3e77df579be24124090cc6641a151a0"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.709837 6441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "8ca76d6564404ff78136d1af7d2a8e3f"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "50f902c3739f75798d8d2021b917b394"
server_key_iv: "d3e77df579be24124090cc6641a151a0"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.713460 6441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:39:48.715811 6458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.717024 6441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:48.717157 6441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "8ca76d6564404ff78136d1af7d2a8e3f"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "50f902c3739f75798d8d2021b917b394"
server_key_iv: "d3e77df579be24124090cc6641a151a0"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.717270 6441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:48.737401 6441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:48.740468 6441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:48.740679 6441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:48.741271 6441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:48.742143 6441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:48.742230 6441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.742309 6441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:48.742352 6441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.751969 6441 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:36525
I20260808 10:39:48.751989 6583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:36525 every 8 connection(s)
I20260808 10:39:48.752594 6441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:48.760690 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6441
I20260808 10:39:48.760848 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:48.761174 1801 external_mini_cluster.cc:1428] Setting key 7ad328e959b55f53a7a70a0b933d99be
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:48.766831 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.754442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:42267 (local address 127.1.194.126:44487)
0808 10:39:48.754611 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:48.754616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:48.755602 (+ 986us) server_negotiation.cc:407] Connection header received
0808 10:39:48.756909 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:48.756912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:48.756957 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:48.757037 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:48.758303 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.758833 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.760006 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.760223 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.760304 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.763248 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:48.763252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:48.763255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:48.763286 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:48.764758 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.765285 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.765290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.765291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.765375 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.765744 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.765748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.765750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.765927 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:48.766035 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:48.766540 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 10:39:48.766662 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:39:48.767715 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.754896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44487 (local address 127.1.194.66:42267)
0808 10:39:48.755379 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:48.755422 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:48.756725 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:48.757188 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:48.757194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:48.757704 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:48.758175 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:48.758187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.758964 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.758967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:48.759890 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:48.759900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.760351 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.760354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:48.760629 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.761377 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:48.761400 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:48.763086 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:48.764882 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:48.764888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:48.764903 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:48.765175 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:48.765469 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:48.765473 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:48.765475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:48.765644 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:48.766042 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:48.766050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:48.766292 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 10:39:48.766472 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 10:39:48.766794 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":119,"threads_started":1}
I20260808 10:39:48.769065 6584 heartbeater.cc:347] Connected to a master server at 127.1.194.126:44487
I20260808 10:39:48.769338 6584 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:48.769867 6584 heartbeater.cc:511] Master 127.1.194.126:44487 requested a full tablet report, sending...
I20260808 10:39:48.771055 6225 ts_manager.cc:194] Registered new tserver with Master: 8ca76d6564404ff78136d1af7d2a8e3f (127.1.194.66:36525)
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:48.771744 6225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:42267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:48.792547 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44487
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:48.903303 6589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:48.903622 6589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:48.903708 6589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:48.907778 6589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:48.907835 6589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:48.907927 6589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:48.913097 6589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:44487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:48.914232 6589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:48.915269 6589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:48.924824 6594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:48.924824 6598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:48.924883 6589 server_base.cc:1034] running on GCE node
W20260808 10:39:48.924827 6596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:48.925396 6589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:48.926122 6589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:48.927291 6589 hybrid_clock.cc:648] HybridClock initialized: now 1786185588927263 us; error 40 us; skew 500 ppm
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:48.930117 6589 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:48.930773 6589 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:48.930833 6589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:48.930994 6589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:48.932977 6589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "ace1cfe7ea674510b634f8aab6fc11c6"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "572c61b316cd8b1967c7edc560ab1fad"
server_key_iv: "d045c202a4afd318cb8aa29a8940e484"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.933588 6589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "ace1cfe7ea674510b634f8aab6fc11c6"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "572c61b316cd8b1967c7edc560ab1fad"
server_key_iv: "d045c202a4afd318cb8aa29a8940e484"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.937086 6589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:48.939438 6607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.940631 6589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:48.940752 6589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "ace1cfe7ea674510b634f8aab6fc11c6"
format_stamp: "Formatted at 2026-08-08 10:39:48 on dist-test-slave-ss4n"
server_key: "572c61b316cd8b1967c7edc560ab1fad"
server_key_iv: "d045c202a4afd318cb8aa29a8940e484"
server_key_version: "encryptionkey@0"
I20260808 10:39:48.940860 6589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:48.951895 6589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:48.954829 6589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:48.955026 6589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:48.955602 6589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:48.956408 6589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:48.956456 6589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.956526 6589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:48.956559 6589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:48.965220 6589 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:44875
I20260808 10:39:48.965305 6725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:44875 every 8 connection(s)
I20260808 10:39:48.965835 6589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:48.968894 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6589
I20260808 10:39:48.969034 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:48.969311 1801 external_mini_cluster.cc:1428] Setting key 7d064b993ce7a1334dedc7ef4a813587
Aug 08 10:39:48 dist-test-slave-ss4n krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185588, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:48.979413 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.967521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:49451 (local address 127.1.194.126:44487)
0808 10:39:48.967656 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:48.967662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:48.968400 (+ 738us) server_negotiation.cc:407] Connection header received
0808 10:39:48.969998 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:48.970002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:48.970072 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:48.970173 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:48.971597 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.972185 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.973325 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.973618 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.973757 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.976132 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:48.976135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:48.976138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:48.976170 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:48.977377 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.977933 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.977937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.977939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.978028 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:48.978370 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:48.978373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:48.978375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:48.978554 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:48.978661 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:48.979129 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 10:39:48.979254 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:39:48.980190 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:48.967817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44487 (local address 127.1.194.67:49451)
0808 10:39:48.968249 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:48.968290 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:48.969789 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:48.970341 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:48.970348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:48.970808 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:48.971452 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:48.971470 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.972311 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.972314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:48.973196 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:48.973205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:48.973793 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:48.973798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:48.973964 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:48.974478 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:48.974493 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:48.975971 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:48.977551 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:48.977558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:48.977571 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:48.977824 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:48.978122 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:48.978126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:48.978128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:48.978265 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:48.978659 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:48.978666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:48.978899 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 10:39:48.979085 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 10:39:48.979415 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260808 10:39:48.981279 6726 heartbeater.cc:347] Connected to a master server at 127.1.194.126:44487
I20260808 10:39:48.981556 6726 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:48.982069 6726 heartbeater.cc:511] Master 127.1.194.126:44487 requested a full tablet report, sending...
I20260808 10:39:48.983203 6231 ts_manager.cc:194] Registered new tserver with Master: ace1cfe7ea674510b634f8aab6fc11c6 (127.1.194.67:44875)
I20260808 10:39:48.983805 6231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:49451
I20260808 10:39:48.997529 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:39:49.009138 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.000094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.1.194.126:44487)
0808 10:39:49.000262 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.000269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.000409 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:39:49.000512 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.000517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.000584 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.000703 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:49.001345 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.002320 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.003146 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.003688 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.003809 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.005241 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:49.005247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:49.005252 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:49.005287 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:49.007113 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:49.007528 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:49.007536 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:49.007539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:49.007647 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:49.007904 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:49.007914 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:49.007918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:49.008301 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:49.008544 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.008706 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.008896 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:39:49.012849 6231 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:49.019070 6231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:49.035461 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.028809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:36525 (local address 127.0.0.1:38020)
0808 10:39:49.029862 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.029878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.029964 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.030482 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.030487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.030508 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.030664 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.030671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.032287 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.032290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.033788 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.033799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.034907 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.034911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.035017 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.035084 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.035174 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.035305 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260808 10:39:49.035909 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.029591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.1.194.65:46633)
0808 10:39:49.030182 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.030187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.030422 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:39:49.030645 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.030650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.030837 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.030953 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.031382 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.032426 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.033945 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.034767 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.034864 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.035015 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.035357 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.035453 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":94,"threads_started":1}
I20260808 10:39:49.036285 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.029370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.1.194.66:36525)
0808 10:39:49.029961 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.029974 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.029998 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:49.030086 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.030091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.030295 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.030809 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.030969 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.032143 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.033927 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.034767 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.035832 (+ 1065us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.035975 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.036030 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.036128 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":186,"threads_started":1}
I20260808 10:39:49.037374 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.029424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:46633 (local address 127.0.0.1:43722)
0808 10:39:49.030327 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.030340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.030487 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.031143 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.031146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.031163 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.031256 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.031261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.032596 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.032598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.033817 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.033827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.035111 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.035114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.035211 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.035230 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.036246 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.037240 (+ 994us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":61,"thread_start_us":203,"threads_started":1}
I20260808 10:39:49.038347 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.028595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44875 (local address 127.0.0.1:58668)
0808 10:39:49.028964 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.029013 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.029142 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.031651 (+ 2509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.031656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.031680 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.031821 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.031829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.034636 (+ 2807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.034642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.036103 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.036118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.037810 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.037816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.037957 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.037995 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.038062 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.038154 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260808 10:39:49.039156 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.028727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.1.194.67:44875)
0808 10:39:49.031150 (+ 2423us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.031162 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.031186 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:49.031260 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.031267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.031463 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.031977 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.032153 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.034448 (+ 2295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.036580 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.037652 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.038574 (+ 922us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.038759 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.038818 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.038944 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2260,"thread_start_us":88,"threads_started":1}
I20260808 10:39:49.039480 6510 tablet_service.cc:1467] Processing CreateTablet for tablet 10ba01c114ea47bfaf612114f1da9fa2 (DEFAULT_TABLE table=test-table [id=e6ea50001cfc4f42b40bee42af7d027b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:49.040102 6365 tablet_service.cc:1467] Processing CreateTablet for tablet 10ba01c114ea47bfaf612114f1da9fa2 (DEFAULT_TABLE table=test-table [id=e6ea50001cfc4f42b40bee42af7d027b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:49.040673 6510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ba01c114ea47bfaf612114f1da9fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:49.041098 6365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ba01c114ea47bfaf612114f1da9fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:49.045285 6658 tablet_service.cc:1467] Processing CreateTablet for tablet 10ba01c114ea47bfaf612114f1da9fa2 (DEFAULT_TABLE table=test-table [id=e6ea50001cfc4f42b40bee42af7d027b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:49.046631 6658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ba01c114ea47bfaf612114f1da9fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:49.049520 6744 tablet_bootstrap.cc:492] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: Bootstrap starting.
I20260808 10:39:49.051666 6744 tablet_bootstrap.cc:654] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:49.052428 6744 log.cc:824] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:49.053172 6745 tablet_bootstrap.cc:492] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Bootstrap starting.
I20260808 10:39:49.054886 6744 tablet_bootstrap.cc:492] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: No bootstrap required, opened a new log
I20260808 10:39:49.055167 6744 ts_tablet_manager.cc:1397] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:39:49.055612 6745 tablet_bootstrap.cc:654] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:49.056916 6745 log.cc:824] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:49.057655 6749 tablet_bootstrap.cc:492] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Bootstrap starting.
I20260808 10:39:49.059779 6749 tablet_bootstrap.cc:654] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:49.060321 6745 tablet_bootstrap.cc:492] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: No bootstrap required, opened a new log
I20260808 10:39:49.060532 6745 ts_tablet_manager.cc:1397] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 10:39:49.061022 6749 log.cc:824] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:49.060928 6744 raft_consensus.cc:348] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.061199 6744 raft_consensus.cc:374] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:49.061251 6744 raft_consensus.cc:729] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c44d8748504e92ae58a8a54127a00a, State: Initialized, Role: FOLLOWER
I20260808 10:39:49.061738 6744 consensus_queue.cc:260] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.063465 6749 tablet_bootstrap.cc:492] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: No bootstrap required, opened a new log
I20260808 10:39:49.063648 6749 ts_tablet_manager.cc:1397] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:39:49.063809 6744 ts_tablet_manager.cc:1428] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260808 10:39:49.064199 6436 heartbeater.cc:503] Master 127.1.194.126:44487 was elected leader, sending a full tablet report...
I20260808 10:39:49.066951 6749 raft_consensus.cc:348] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.067237 6749 raft_consensus.cc:374] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:49.067296 6749 raft_consensus.cc:729] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace1cfe7ea674510b634f8aab6fc11c6, State: Initialized, Role: FOLLOWER
I20260808 10:39:49.067899 6749 consensus_queue.cc:260] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.068254 6745 raft_consensus.cc:348] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.068547 6726 heartbeater.cc:503] Master 127.1.194.126:44487 was elected leader, sending a full tablet report...
I20260808 10:39:49.068595 6745 raft_consensus.cc:374] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:49.068670 6745 raft_consensus.cc:729] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ca76d6564404ff78136d1af7d2a8e3f, State: Initialized, Role: FOLLOWER
I20260808 10:39:49.068945 6749 ts_tablet_manager.cc:1428] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:39:49.069293 6745 consensus_queue.cc:260] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.070619 6745 ts_tablet_manager.cc:1428] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260808 10:39:49.071055 6584 heartbeater.cc:503] Master 127.1.194.126:44487 was elected leader, sending a full tablet report...
W20260808 10:39:49.216915 6727 tablet.cc:2369] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:49.253768 6585 tablet.cc:2369] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:49.275259 6437 tablet.cc:2369] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:49.346738 6753 raft_consensus.cc:482] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:49.346941 6753 raft_consensus.cc:504] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.348078 6753 leader_election.cc:288] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c44d8748504e92ae58a8a54127a00a (127.1.194.65:46633), 8ca76d6564404ff78136d1af7d2a8e3f (127.1.194.66:36525)
I20260808 10:39:49.349895 6754 raft_consensus.cc:482] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:49.350128 6754 raft_consensus.cc:504] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.352589 6754 leader_election.cc:288] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace1cfe7ea674510b634f8aab6fc11c6 (127.1.194.67:44875), e5c44d8748504e92ae58a8a54127a00a (127.1.194.65:46633)
I20260808 10:39:49.353024 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.348470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:59373 (local address 127.1.194.66:36525)
0808 10:39:49.348606 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.348612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.349020 (+ 408us) server_negotiation.cc:407] Connection header received
0808 10:39:49.349100 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.349103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.349149 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.349227 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.349699 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.350321 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.351476 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.352208 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.352312 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.352361 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.352803 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.352899 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:39:49.353663 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.348397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:36525 (local address 127.1.194.67:59373)
0808 10:39:49.348870 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.348883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.349006 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.349327 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.349330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.349356 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.349501 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.349541 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.350444 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.350449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.351319 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.351332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.352458 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.352461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.352577 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.352599 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.353372 (+ 773us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.353458 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":33,"thread_start_us":126,"threads_started":1}
I20260808 10:39:49.353678 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.348470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:40953 (local address 127.1.194.65:46633)
0808 10:39:49.348606 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.348612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.348715 (+ 103us) server_negotiation.cc:407] Connection header received
0808 10:39:49.348862 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.348865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.348929 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.349014 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.349377 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.350373 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.352043 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.352758 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.352905 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.352980 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.353402 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.353528 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:39:49.354084 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.348397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:46633 (local address 127.1.194.67:40953)
0808 10:39:49.348614 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.348630 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.348747 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.349061 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.349067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.349088 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.349225 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.349232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.350559 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.350563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.351787 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.351804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.353048 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.353053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.353196 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.353225 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.353305 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.353934 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":531}
I20260808 10:39:49.354467 6533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ba01c114ea47bfaf612114f1da9fa2" candidate_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" is_pre_election: true
I20260808 10:39:49.354691 6533 raft_consensus.cc:2454] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace1cfe7ea674510b634f8aab6fc11c6 in term 0.
I20260808 10:39:49.354828 6387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ba01c114ea47bfaf612114f1da9fa2" candidate_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c44d8748504e92ae58a8a54127a00a" is_pre_election: true
I20260808 10:39:49.355151 6387 raft_consensus.cc:2454] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace1cfe7ea674510b634f8aab6fc11c6 in term 0.
I20260808 10:39:49.355381 6611 leader_election.cc:302] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ca76d6564404ff78136d1af7d2a8e3f, ace1cfe7ea674510b634f8aab6fc11c6; no voters:
I20260808 10:39:49.355681 6753 raft_consensus.cc:2781] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:49.355751 6753 raft_consensus.cc:482] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:49.355794 6753 raft_consensus.cc:3037] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:49.357139 6753 raft_consensus.cc:504] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.357548 6753 leader_election.cc:288] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [CANDIDATE]: Term 1 election: Requested vote from peers e5c44d8748504e92ae58a8a54127a00a (127.1.194.65:46633), 8ca76d6564404ff78136d1af7d2a8e3f (127.1.194.66:36525)
I20260808 10:39:49.357614 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.352302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:46633 (local address 127.1.194.66:34479)
0808 10:39:49.353277 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.353291 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.353415 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.353837 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.353840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.353855 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.353948 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.353952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.354925 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.354931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.356183 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.356193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.357040 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.357044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.357166 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.357194 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.357288 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.357365 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":4,"thread_start_us":79,"threads_started":1}
I20260808 10:39:49.357728 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.352502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:36695 (local address 127.1.194.67:44875)
0808 10:39:49.352615 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.352621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.352953 (+ 332us) server_negotiation.cc:407] Connection header received
0808 10:39:49.353175 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.353178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.353227 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.353286 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.353807 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.354615 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.356413 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.356962 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.357102 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.357155 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.357488 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.357586 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":52}
I20260808 10:39:49.357921 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.352718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:34479 (local address 127.1.194.65:46633)
0808 10:39:49.353092 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.353102 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.353350 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:39:49.353565 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.353569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.353612 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.353668 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:49.354096 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.354786 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.356338 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.356881 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.357050 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.357107 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.357374 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.357448 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":114,"threads_started":1}
I20260808 10:39:49.357960 6387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ba01c114ea47bfaf612114f1da9fa2" candidate_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c44d8748504e92ae58a8a54127a00a"
I20260808 10:39:49.358155 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.351701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44875 (local address 127.1.194.66:36695)
0808 10:39:49.352852 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.352869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.353039 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.353501 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.353506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.353528 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:49.353648 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.353656 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.354789 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.354793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.356235 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:49.356278 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.357189 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.357194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:49.357324 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.357348 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.357869 (+ 521us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.357971 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"mutex_wait_us":120,"thread_start_us":113,"threads_started":1}
I20260808 10:39:49.358173 6387 raft_consensus.cc:3037] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:49.358433 6386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ba01c114ea47bfaf612114f1da9fa2" candidate_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c44d8748504e92ae58a8a54127a00a" is_pre_election: true
I20260808 10:39:49.358737 6679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ba01c114ea47bfaf612114f1da9fa2" candidate_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" is_pre_election: true
I20260808 10:39:49.359007 6679 raft_consensus.cc:2379] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ca76d6564404ff78136d1af7d2a8e3f in current term 1: Already voted for candidate ace1cfe7ea674510b634f8aab6fc11c6 in this term.
I20260808 10:39:49.359185 6533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ba01c114ea47bfaf612114f1da9fa2" candidate_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ca76d6564404ff78136d1af7d2a8e3f"
I20260808 10:39:49.359293 6533 raft_consensus.cc:3037] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:49.359536 6387 raft_consensus.cc:2454] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace1cfe7ea674510b634f8aab6fc11c6 in term 1.
I20260808 10:39:49.359598 6461 leader_election.cc:302] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ca76d6564404ff78136d1af7d2a8e3f; no voters: ace1cfe7ea674510b634f8aab6fc11c6, e5c44d8748504e92ae58a8a54127a00a
I20260808 10:39:49.359879 6610 leader_election.cc:302] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace1cfe7ea674510b634f8aab6fc11c6, e5c44d8748504e92ae58a8a54127a00a; no voters:
I20260808 10:39:49.360036 6753 raft_consensus.cc:2781] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:49.360239 6753 raft_consensus.cc:686] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 1 LEADER]: Becoming Leader. State: Replica: ace1cfe7ea674510b634f8aab6fc11c6, State: Running, Role: LEADER
I20260808 10:39:49.360649 6533 raft_consensus.cc:2454] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace1cfe7ea674510b634f8aab6fc11c6 in term 1.
I20260808 10:39:49.360592 6753 consensus_queue.cc:237] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } }
I20260808 10:39:49.360776 6754 raft_consensus.cc:2726] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 10:39:49.364037 6225 catalog_manager.cc:5495] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ace1cfe7ea674510b634f8aab6fc11c6 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1cfe7ea674510b634f8aab6fc11c6" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:49.404351 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.400720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.1.194.67:44875)
0808 10:39:49.400852 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.400856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.400874 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:39:49.401012 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.401014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.401070 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.401156 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:49.401405 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.401994 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.402910 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:49.403118 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:49.403196 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:49.403527 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:49.403938 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:49.404030 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.404177 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.404221 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:39:49.410542 6387 raft_consensus.cc:1261] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a [term 1 FOLLOWER]: Refusing update from remote peer ace1cfe7ea674510b634f8aab6fc11c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:49.410599 6533 raft_consensus.cc:1261] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 1 FOLLOWER]: Refusing update from remote peer ace1cfe7ea674510b634f8aab6fc11c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:49.411312 6753 consensus_queue.cc:1035] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ca76d6564404ff78136d1af7d2a8e3f" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:49.411813 6759 consensus_queue.cc:1035] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c44d8748504e92ae58a8a54127a00a" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:49.420866 6763 mvcc.cc:205] Tried to move back new op lower bound from 7316216174218002432 to 7316216174023806976. Current Snapshot: MvccSnapshot[applied={T|T < 7316216174218002432}]
I20260808 10:39:49.429831 6225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:49.430135 6225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:49.432732 6225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b61f87fdf6b64376acc077cc0305e452: Sending DeleteTablet for 3 replicas of tablet 10ba01c114ea47bfaf612114f1da9fa2
I20260808 10:39:49.433660 6365 tablet_service.cc:1514] Processing DeleteTablet for tablet 10ba01c114ea47bfaf612114f1da9fa2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:49 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:43722
I20260808 10:39:49.433691 6510 tablet_service.cc:1514] Processing DeleteTablet for tablet 10ba01c114ea47bfaf612114f1da9fa2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:49 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:38020
I20260808 10:39:49.434208 6773 tablet_replica.cc:330] T 10ba01c114ea47bfaf612114f1da9fa2 P e5c44d8748504e92ae58a8a54127a00a: stopping tablet replica
I20260808 10:39:49.434233 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6297
I20260808 10:39:49.434230 6658 tablet_service.cc:1514] Processing DeleteTablet for tablet 10ba01c114ea47bfaf612114f1da9fa2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:49 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:58668
I20260808 10:39:49.435825 6775 tablet_replica.cc:330] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: stopping tablet replica
I20260808 10:39:49.436234 6775 raft_consensus.cc:2229] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:49.436764 6775 raft_consensus.cc:2258] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:49.437237 6774 tablet_replica.cc:330] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: stopping tablet replica
I20260808 10:39:49.437604 6774 raft_consensus.cc:2229] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:49.437949 6774 raft_consensus.cc:2258] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:49.439678 6775 ts_tablet_manager.cc:1905] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:49.442780 6775 ts_tablet_manager.cc:1918] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:49.442888 6775 log.cc:1196] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/10ba01c114ea47bfaf612114f1da9fa2
I20260808 10:39:49.443091 6774 ts_tablet_manager.cc:1905] T 10ba01c114ea47bfaf612114f1da9fa2 P 8ca76d6564404ff78136d1af7d2a8e3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:49.443251 6775 ts_tablet_manager.cc:1939] T 10ba01c114ea47bfaf612114f1da9fa2 P ace1cfe7ea674510b634f8aab6fc11c6: Deleting consensus metadata
W20260808 10:39:49.444257 6216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:46633 (error 108)
I20260808 10:39:49.444484 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6441
I20260808 10:39:49.450124 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.449816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:46633 (local address 127.0.0.1:43734)
0808 10:39:49.449928 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.450013 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:39:49.450312 6216 catalog_manager.cc:4584] TS e5c44d8748504e92ae58a8a54127a00a (127.1.194.65:46633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10ba01c114ea47bfaf612114f1da9fa2: Network error: Client connection negotiation failed: client connection to 127.1.194.65:46633: connect: Connection refused (error 111)
I20260808 10:39:49.451056 6216 catalog_manager.cc:4841] TS ace1cfe7ea674510b634f8aab6fc11c6 (127.1.194.67:44875): tablet 10ba01c114ea47bfaf612114f1da9fa2 (table test-table [id=e6ea50001cfc4f42b40bee42af7d027b]) successfully deleted
I20260808 10:39:49.455113 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6589
I20260808 10:39:49.457451 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.455229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:36525 (local address 127.0.0.1:38036)
0808 10:39:49.457271 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.457343 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.66:36525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1933}
I20260808 10:39:49.462775 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6196
2026-08-08T10:39:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2731 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T10:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:49Z Disabled control of system clock
I20260808 10:39:49.496096 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:39621
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:39621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:39:49.610996 6782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:49.611303 6782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:49.611341 6782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:49.615393 6782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:49.615464 6782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:49.615481 6782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:49.615499 6782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:49.615514 6782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:49.621253 6782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:39621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:39621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:49.622411 6782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:49.623871 6782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:49.632742 6790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:49.632745 6787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:49.632714 6788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:49.633054 6782 server_base.cc:1034] running on GCE node
I20260808 10:39:49.633644 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:49.634713 6782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:49.635882 6782 hybrid_clock.cc:648] HybridClock initialized: now 1786185589635851 us; error 43 us; skew 500 ppm
I20260808 10:39:49.637809 6782 webserver.cc:460] Webserver started at http://127.1.194.126:32931/ using document root <none> and password file <none>
I20260808 10:39:49.638335 6782 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:49.638393 6782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:49.638562 6782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:49.640570 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "64b7010ac6a44ef4b3704efa890e7fff"
format_stamp: "Formatted at 2026-08-08 10:39:49 on dist-test-slave-ss4n"
server_key: "7eeb89a7ec57529ed13c3370fd506055"
server_key_iv: "b7ff24a8cdec246860d85f6ed2b52d63"
server_key_version: "encryptionkey@0"
I20260808 10:39:49.641086 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "64b7010ac6a44ef4b3704efa890e7fff"
format_stamp: "Formatted at 2026-08-08 10:39:49 on dist-test-slave-ss4n"
server_key: "7eeb89a7ec57529ed13c3370fd506055"
server_key_iv: "b7ff24a8cdec246860d85f6ed2b52d63"
server_key_version: "encryptionkey@0"
I20260808 10:39:49.644804 6782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:39:49.647195 6796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:49.648373 6782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:49.648473 6782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "64b7010ac6a44ef4b3704efa890e7fff"
format_stamp: "Formatted at 2026-08-08 10:39:49 on dist-test-slave-ss4n"
server_key: "7eeb89a7ec57529ed13c3370fd506055"
server_key_iv: "b7ff24a8cdec246860d85f6ed2b52d63"
server_key_version: "encryptionkey@0"
I20260808 10:39:49.648613 6782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:49.653154 6782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:49.653787 6782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:49.653924 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:49.660696 6782 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:39621
I20260808 10:39:49.660741 6857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:39621 every 8 connection(s)
I20260808 10:39:49.661689 6782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:49.662405 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6782
I20260808 10:39:49.662545 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:49.662811 1801 external_mini_cluster.cc:1428] Setting key 54c1a38dc67d78b4fb16195ad77a4a7f
I20260808 10:39:49.664920 6858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:49.668888 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.664233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50606 (local address 127.1.194.126:39621)
0808 10:39:49.665446 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.665459 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.665547 (+ 88us) server_negotiation.cc:407] Connection header received
0808 10:39:49.666194 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.666210 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.666528 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.666849 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:49.667711 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:49.667715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:49.667725 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:49.667735 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:49.667787 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:49.667803 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:49.667951 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.668031 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.668259 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":704,"threads_started":1}
I20260808 10:39:49.671152 6858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff: Bootstrap starting.
I20260808 10:39:49.673192 6858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:49.674029 6858 log.cc:824] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:49.676029 6858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff: No bootstrap required, opened a new log
I20260808 10:39:49.679803 6858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b7010ac6a44ef4b3704efa890e7fff" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39621 } }
I20260808 10:39:49.680027 6858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:49.680063 6858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b7010ac6a44ef4b3704efa890e7fff, State: Initialized, Role: FOLLOWER
I20260808 10:39:49.680428 6858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b7010ac6a44ef4b3704efa890e7fff" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39621 } }
I20260808 10:39:49.680541 6858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:49.680585 6858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:49.680653 6858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:49.681576 6858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b7010ac6a44ef4b3704efa890e7fff" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39621 } }
I20260808 10:39:49.681857 6858 leader_election.cc:302] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64b7010ac6a44ef4b3704efa890e7fff; no voters:
I20260808 10:39:49.682109 6858 leader_election.cc:288] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:49.682255 6863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:49.682493 6863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [term 1 LEADER]: Becoming Leader. State: Replica: 64b7010ac6a44ef4b3704efa890e7fff, State: Running, Role: LEADER
I20260808 10:39:49.682804 6863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b7010ac6a44ef4b3704efa890e7fff" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39621 } }
I20260808 10:39:49.683369 6858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:49.684540 6865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64b7010ac6a44ef4b3704efa890e7fff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b7010ac6a44ef4b3704efa890e7fff" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39621 } } }
I20260808 10:39:49.684667 6865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:49.684950 6866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64b7010ac6a44ef4b3704efa890e7fff. Latest consensus state: current_term: 1 leader_uuid: "64b7010ac6a44ef4b3704efa890e7fff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b7010ac6a44ef4b3704efa890e7fff" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39621 } } }
I20260808 10:39:49.685043 6866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:49.685164 6873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:49.688228 6873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:49.694418 6873 catalog_manager.cc:1347] Generated new cluster ID: a4cf02e58d574c0d982a5179deb13a27
I20260808 10:39:49.694491 6873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:49.712589 6873 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:49.712967 6873 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:49.719825 6873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff: Generated new TSK 0
I20260808 10:39:49.720520 6873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:49.727731 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39621
--builtin_ntp_servers=127.1.194.84:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:39:49.846953 6884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:49.847249 6884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:49.847289 6884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:49.851424 6884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:49.851486 6884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:49.851599 6884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:49.856627 6884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:49.857807 6884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:49.859184 6884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:49.868889 6892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:49.868898 6890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:49.868917 6889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:49.869065 6884 server_base.cc:1034] running on GCE node
I20260808 10:39:49.869732 6884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:49.870414 6884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:49.871585 6884 hybrid_clock.cc:648] HybridClock initialized: now 1786185589871557 us; error 42 us; skew 500 ppm
I20260808 10:39:49.873585 6884 webserver.cc:460] Webserver started at http://127.1.194.65:34653/ using document root <none> and password file <none>
I20260808 10:39:49.874146 6884 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:49.874208 6884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:49.874481 6884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:49.876526 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "33fbc20d8fd5450b8553477b02d3df2b"
format_stamp: "Formatted at 2026-08-08 10:39:49 on dist-test-slave-ss4n"
server_key: "46685c80467b472f08c0f4eb93ab3ab7"
server_key_iv: "00897a446c3968eae0ea9538bb9c3ed2"
server_key_version: "encryptionkey@0"
I20260808 10:39:49.877067 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "33fbc20d8fd5450b8553477b02d3df2b"
format_stamp: "Formatted at 2026-08-08 10:39:49 on dist-test-slave-ss4n"
server_key: "46685c80467b472f08c0f4eb93ab3ab7"
server_key_iv: "00897a446c3968eae0ea9538bb9c3ed2"
server_key_version: "encryptionkey@0"
I20260808 10:39:49.880672 6884 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:39:49.883056 6898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:49.884299 6884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:49.884418 6884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "33fbc20d8fd5450b8553477b02d3df2b"
format_stamp: "Formatted at 2026-08-08 10:39:49 on dist-test-slave-ss4n"
server_key: "46685c80467b472f08c0f4eb93ab3ab7"
server_key_iv: "00897a446c3968eae0ea9538bb9c3ed2"
server_key_version: "encryptionkey@0"
I20260808 10:39:49.884572 6884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:49.889021 6884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:49.889642 6884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:49.889814 6884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:49.890336 6884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:49.891212 6884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:49.891260 6884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:49.891297 6884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:49.891340 6884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:49.900002 6884 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:43179
I20260808 10:39:49.900043 7020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:43179 every 8 connection(s)
I20260808 10:39:49.900872 6884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:49.905081 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 6884
I20260808 10:39:49.905225 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:49.905596 1801 external_mini_cluster.cc:1428] Setting key 6c4276aa6c516d0522eadec1b981109d
I20260808 10:39:49.907370 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.902729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:53031 (local address 127.1.194.126:39621)
0808 10:39:49.902874 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:49.902878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:49.903911 (+ 1033us) server_negotiation.cc:407] Connection header received
0808 10:39:49.905651 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:49.905656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:49.905709 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:49.905806 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:49.906932 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:49.906938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:49.906941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:49.906954 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:49.906975 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:49.906985 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:49.907075 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:49.907234 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 10:39:49.907271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:39:49.907680 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39621
--builtin_ntp_servers=127.1.194.84:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 10:39:49.908548 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:49.903072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39621 (local address 127.1.194.65:53031)
0808 10:39:49.903756 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:49.903796 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:49.905350 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:49.905988 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:49.905996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:49.906098 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:49.906711 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:49.906713 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:49.906752 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:49.906823 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:49.907079 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:49.907087 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:49.907093 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:39:49.907267 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 10:39:49.907530 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260808 10:39:49.909772 7021 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39621
I20260808 10:39:49.909999 7021 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:49.910398 7021 heartbeater.cc:511] Master 127.1.194.126:39621 requested a full tablet report, sending...
I20260808 10:39:49.911872 6816 ts_manager.cc:194] Registered new tserver with Master: 33fbc20d8fd5450b8553477b02d3df2b (127.1.194.65:43179)
W20260808 10:39:50.023272 7025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:50.023524 7025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:50.023562 7025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:50.027822 7025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:50.027876 7025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:50.027964 7025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:50.032644 7025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:50.033751 7025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:50.035156 7025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:50.044854 7033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:50.044854 7031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:50.044950 7030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:50.045097 7025 server_base.cc:1034] running on GCE node
I20260808 10:39:50.045655 7025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:50.046406 7025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:50.047602 7025 hybrid_clock.cc:648] HybridClock initialized: now 1786185590047568 us; error 50 us; skew 500 ppm
I20260808 10:39:50.049721 7025 webserver.cc:460] Webserver started at http://127.1.194.66:33457/ using document root <none> and password file <none>
I20260808 10:39:50.050334 7025 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:50.050400 7025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:50.050686 7025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:50.052975 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "46d4901973984f6fbb1df5210ef8119c"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "e51b62d8fc97db9bb208853ac3f4e960"
server_key_iv: "930617f8749817db6f4067a886d4921e"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.053624 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "46d4901973984f6fbb1df5210ef8119c"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "e51b62d8fc97db9bb208853ac3f4e960"
server_key_iv: "930617f8749817db6f4067a886d4921e"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.057345 7025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:39:50.060258 7040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.061697 7025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:50.061825 7025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "46d4901973984f6fbb1df5210ef8119c"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "e51b62d8fc97db9bb208853ac3f4e960"
server_key_iv: "930617f8749817db6f4067a886d4921e"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.061932 7025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:50.066475 7025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:50.067066 7025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:50.067245 7025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:50.067782 7025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:50.068672 7025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:50.068718 7025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.068780 7025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:50.068813 7025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.077785 7025 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:43593
I20260808 10:39:50.077813 7160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:43593 every 8 connection(s)
I20260808 10:39:50.078667 7025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:50.085103 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.080393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:48521 (local address 127.1.194.126:39621)
0808 10:39:50.080652 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.080657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.081745 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 10:39:50.083004 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.083007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.083048 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.083127 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.084502 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.084505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.084507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.084517 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.084584 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.084593 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.084674 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.084841 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.084886 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:39:50.085120 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7025
I20260808 10:39:50.085235 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:50.085624 1801 external_mini_cluster.cc:1428] Setting key cf3148f2d6bdf1b19822af10e9dec34a
I20260808 10:39:50.086138 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.080834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39621 (local address 127.1.194.66:48521)
0808 10:39:50.081592 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.081643 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.082817 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.083404 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.083410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.083491 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.084181 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.084183 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.084241 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.084336 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.084676 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.084683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.084687 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.084846 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.085133 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260808 10:39:50.087337 7161 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39621
I20260808 10:39:50.087589 7161 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:50.087855 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39621
--builtin_ntp_servers=127.1.194.84:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 10:39:50.088020 7161 heartbeater.cc:511] Master 127.1.194.126:39621 requested a full tablet report, sending...
I20260808 10:39:50.089191 6815 ts_manager.cc:194] Registered new tserver with Master: 46d4901973984f6fbb1df5210ef8119c (127.1.194.66:43593)
W20260808 10:39:50.210696 7164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:50.210988 7164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:50.211040 7164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:50.215351 7164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:50.215411 7164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:50.215469 7164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:50.220474 7164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:50.221729 7164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:50.223124 7164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:50.232605 7172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:50.232725 7170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:50.232621 7169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:50.232949 7164 server_base.cc:1034] running on GCE node
I20260808 10:39:50.233366 7164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:50.234052 7164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:50.235239 7164 hybrid_clock.cc:648] HybridClock initialized: now 1786185590235212 us; error 44 us; skew 500 ppm
I20260808 10:39:50.237116 7164 webserver.cc:460] Webserver started at http://127.1.194.67:46615/ using document root <none> and password file <none>
I20260808 10:39:50.237780 7164 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:50.237875 7164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:50.238106 7164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:50.240105 7164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "4f8fbad4e1904423a280555c72660dbb"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "96baab0827aa6b220a7e3e6e5d7e9fa5"
server_key_iv: "1d07ee2e953da908ea6ad3c426ed6ab8"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.240702 7164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "4f8fbad4e1904423a280555c72660dbb"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "96baab0827aa6b220a7e3e6e5d7e9fa5"
server_key_iv: "1d07ee2e953da908ea6ad3c426ed6ab8"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.244479 7164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:39:50.247087 7178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.248337 7164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:50.248574 7164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "4f8fbad4e1904423a280555c72660dbb"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "96baab0827aa6b220a7e3e6e5d7e9fa5"
server_key_iv: "1d07ee2e953da908ea6ad3c426ed6ab8"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.248715 7164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:50.253947 7164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:50.254549 7164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:50.254739 7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:50.255296 7164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:50.256191 7164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:50.256263 7164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.256335 7164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:50.256382 7164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.265127 7164 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:36163
I20260808 10:39:50.265197 7304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:36163 every 8 connection(s)
I20260808 10:39:50.266218 7164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:50.272594 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.268187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:44831 (local address 127.1.194.126:39621)
0808 10:39:50.268352 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.268356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.269378 (+ 1022us) server_negotiation.cc:407] Connection header received
0808 10:39:50.270823 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.270826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.270873 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.270955 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.272104 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.272107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.272110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.272122 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.272146 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.272155 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.272261 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.272403 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.272449 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:39:50.273393 7308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.268585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39621 (local address 127.1.194.67:44831)
0808 10:39:50.269234 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.269273 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.270611 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.271131 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.271137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.271220 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.271845 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.271847 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.271886 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.271950 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.272254 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.272262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.272269 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.272445 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.272676 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260808 10:39:50.274533 7305 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39621
I20260808 10:39:50.274776 7305 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:50.275182 7305 heartbeater.cc:511] Master 127.1.194.126:39621 requested a full tablet report, sending...
I20260808 10:39:50.275511 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7164
I20260808 10:39:50.275667 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:50.276003 1801 external_mini_cluster.cc:1428] Setting key bc9081220d804108205414447754b58f
I20260808 10:39:50.276243 6815 ts_manager.cc:194] Registered new tserver with Master: 4f8fbad4e1904423a280555c72660dbb (127.1.194.67:36163)
I20260808 10:39:50.278048 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:39:50.281585 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.279909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.1.194.126:39621)
0808 10:39:50.280052 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.280058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.280224 (+ 166us) server_negotiation.cc:407] Connection header received
0808 10:39:50.280369 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.280372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.280407 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.280496 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.281119 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.281123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.281125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.281136 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.281154 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.281163 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.281256 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.281348 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.281388 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:39:50.284206 6816 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:50.287245 6816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:50.298956 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.295938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.1.194.66:43593)
0808 10:39:50.296407 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.296426 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.296469 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:39:50.296528 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.296533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.296732 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.296891 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.298403 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.298409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.298421 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.298432 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.298476 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.298496 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.298610 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.298722 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.298792 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":140,"threads_started":1}
I20260808 10:39:50.298947 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.295798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:43593 (local address 127.0.0.1:37778)
0808 10:39:50.296160 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.296211 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.296313 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.296884 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.296889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.296962 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.298203 (+ 1241us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.298205 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.298219 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.298278 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.298592 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.298601 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.298606 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.298710 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.298776 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
I20260808 10:39:50.298990 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.295798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:43179 (local address 127.0.0.1:49078)
0808 10:39:50.296588 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.296602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.296669 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.297566 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.297569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.297586 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.298196 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.298198 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.298220 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.298281 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.298687 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.298690 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.298694 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.298784 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.298829 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":524,"thread_start_us":103,"threads_started":1}
I20260808 10:39:50.299906 7322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.296046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.1.194.65:43179)
0808 10:39:50.296727 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.296739 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.296764 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:39:50.296824 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.296829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.297012 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.297159 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.298433 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.298439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.298451 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.298464 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.298507 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.298538 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.298640 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.299656 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.299731 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":143,"threads_started":1}
I20260808 10:39:50.300209 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.296590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36163 (local address 127.0.0.1:43262)
0808 10:39:50.297844 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.297859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.297940 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.299087 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.299093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.299113 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.299563 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.299565 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.299575 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.299605 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.299940 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.299945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.299953 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.300048 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.300086 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1173,"thread_start_us":90,"threads_started":1}
I20260808 10:39:50.300696 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.296998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.1.194.67:36163)
0808 10:39:50.297700 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.297715 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.298037 (+ 322us) server_negotiation.cc:407] Connection header received
0808 10:39:50.298097 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.298116 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.298313 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.298448 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.299736 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.299742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.299752 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.299764 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.299804 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.299831 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.299932 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.300428 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.300512 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":170,"threads_started":1}
I20260808 10:39:50.302223 7089 tablet_service.cc:1467] Processing CreateTablet for tablet 80c94ceb5ddd4e5d8132a9c200e36003 (DEFAULT_TABLE table=test-table [id=415afb52bb7d4f70996a1e7c6aab032f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:50.303159 7089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80c94ceb5ddd4e5d8132a9c200e36003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:50.302733 6951 tablet_service.cc:1467] Processing CreateTablet for tablet 80c94ceb5ddd4e5d8132a9c200e36003 (DEFAULT_TABLE table=test-table [id=415afb52bb7d4f70996a1e7c6aab032f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:50.303287 7231 tablet_service.cc:1467] Processing CreateTablet for tablet 80c94ceb5ddd4e5d8132a9c200e36003 (DEFAULT_TABLE table=test-table [id=415afb52bb7d4f70996a1e7c6aab032f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:50.303783 6951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80c94ceb5ddd4e5d8132a9c200e36003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:50.304306 7231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80c94ceb5ddd4e5d8132a9c200e36003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:50.309422 7327 tablet_bootstrap.cc:492] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: Bootstrap starting.
I20260808 10:39:50.311235 7328 tablet_bootstrap.cc:492] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: Bootstrap starting.
I20260808 10:39:50.311842 7327 tablet_bootstrap.cc:654] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:50.312245 7329 tablet_bootstrap.cc:492] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Bootstrap starting.
I20260808 10:39:50.312919 7327 log.cc:824] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:50.313417 7328 tablet_bootstrap.cc:654] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:50.314247 7328 log.cc:824] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:50.314373 7329 tablet_bootstrap.cc:654] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:50.315177 7327 tablet_bootstrap.cc:492] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: No bootstrap required, opened a new log
I20260808 10:39:50.315248 7329 log.cc:824] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:50.315369 7327 ts_tablet_manager.cc:1397] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:39:50.316207 7328 tablet_bootstrap.cc:492] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: No bootstrap required, opened a new log
I20260808 10:39:50.316383 7328 ts_tablet_manager.cc:1397] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:39:50.317030 7329 tablet_bootstrap.cc:492] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: No bootstrap required, opened a new log
I20260808 10:39:50.317230 7329 ts_tablet_manager.cc:1397] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:39:50.320071 7327 raft_consensus.cc:348] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.320356 7327 raft_consensus.cc:374] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:50.320415 7327 raft_consensus.cc:729] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d4901973984f6fbb1df5210ef8119c, State: Initialized, Role: FOLLOWER
I20260808 10:39:50.320775 7328 raft_consensus.cc:348] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.320873 7327 consensus_queue.cc:260] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.321012 7328 raft_consensus.cc:374] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:50.321077 7328 raft_consensus.cc:729] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fbc20d8fd5450b8553477b02d3df2b, State: Initialized, Role: FOLLOWER
I20260808 10:39:50.321287 7329 raft_consensus.cc:348] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.321508 7328 consensus_queue.cc:260] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.321587 7329 raft_consensus.cc:374] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:50.321646 7329 raft_consensus.cc:729] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8fbad4e1904423a280555c72660dbb, State: Initialized, Role: FOLLOWER
I20260808 10:39:50.321938 7327 ts_tablet_manager.cc:1428] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 10:39:50.322206 7161 heartbeater.cc:503] Master 127.1.194.126:39621 was elected leader, sending a full tablet report...
I20260808 10:39:50.322408 7329 consensus_queue.cc:260] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.323072 7305 heartbeater.cc:503] Master 127.1.194.126:39621 was elected leader, sending a full tablet report...
I20260808 10:39:50.323410 7329 ts_tablet_manager.cc:1428] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:39:50.323451 7328 ts_tablet_manager.cc:1428] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 10:39:50.323731 7021 heartbeater.cc:503] Master 127.1.194.126:39621 was elected leader, sending a full tablet report...
W20260808 10:39:50.329643 7162 tablet.cc:2369] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:50.402073 7022 tablet.cc:2369] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:50.412549 7335 raft_consensus.cc:482] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:50.412777 7335 raft_consensus.cc:504] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.413990 7335 leader_election.cc:288] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33fbc20d8fd5450b8553477b02d3df2b (127.1.194.65:43179), 4f8fbad4e1904423a280555c72660dbb (127.1.194.67:36163)
I20260808 10:39:50.416378 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.414551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:51021 (local address 127.1.194.67:36163)
0808 10:39:50.414677 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.414681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.414830 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:39:50.414965 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.414968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.415010 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.415107 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.415912 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.415915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.415918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.415928 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.415948 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.415956 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.416033 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.416198 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.416240 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:39:50.416400 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.414376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36163 (local address 127.1.194.66:51021)
0808 10:39:50.414714 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.414736 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.414855 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.415128 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.415130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.415145 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.415749 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.415751 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.415761 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.415786 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.416061 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.416063 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.416066 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.416184 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.416226 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":118,"threads_started":1}
I20260808 10:39:50.416404 7322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.414348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:42083 (local address 127.1.194.65:43179)
0808 10:39:50.414487 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.414492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.414511 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:39:50.414720 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.414724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.414762 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.414861 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.415715 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.415719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.415726 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.415750 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.415774 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.415784 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.415892 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.416221 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.416267 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:39:50.416404 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.414215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:43179 (local address 127.1.194.66:42083)
0808 10:39:50.414373 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.414392 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.414494 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.414881 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.414884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.414900 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:50.415467 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:39:50.415471 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:50.415486 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:39:50.415526 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:50.416076 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:50.416078 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:50.416081 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:39:50.416211 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 10:39:50.416252 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:39:50.417145 7254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c94ceb5ddd4e5d8132a9c200e36003" candidate_uuid: "46d4901973984f6fbb1df5210ef8119c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8fbad4e1904423a280555c72660dbb" is_pre_election: true
I20260808 10:39:50.417377 6973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c94ceb5ddd4e5d8132a9c200e36003" candidate_uuid: "46d4901973984f6fbb1df5210ef8119c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fbc20d8fd5450b8553477b02d3df2b" is_pre_election: true
I20260808 10:39:50.417465 7254 raft_consensus.cc:2454] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46d4901973984f6fbb1df5210ef8119c in term 0.
I20260808 10:39:50.417670 6973 raft_consensus.cc:2454] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46d4901973984f6fbb1df5210ef8119c in term 0.
I20260808 10:39:50.418030 7041 leader_election.cc:302] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46d4901973984f6fbb1df5210ef8119c, 4f8fbad4e1904423a280555c72660dbb; no voters:
I20260808 10:39:50.418316 7335 raft_consensus.cc:2781] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:50.418401 7335 raft_consensus.cc:482] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:50.418445 7335 raft_consensus.cc:3037] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:50.419481 7335 raft_consensus.cc:504] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.419804 7335 leader_election.cc:288] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [CANDIDATE]: Term 1 election: Requested vote from peers 33fbc20d8fd5450b8553477b02d3df2b (127.1.194.65:43179), 4f8fbad4e1904423a280555c72660dbb (127.1.194.67:36163)
I20260808 10:39:50.420118 6973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c94ceb5ddd4e5d8132a9c200e36003" candidate_uuid: "46d4901973984f6fbb1df5210ef8119c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fbc20d8fd5450b8553477b02d3df2b"
I20260808 10:39:50.420161 7254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c94ceb5ddd4e5d8132a9c200e36003" candidate_uuid: "46d4901973984f6fbb1df5210ef8119c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8fbad4e1904423a280555c72660dbb"
I20260808 10:39:50.420250 6973 raft_consensus.cc:3037] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:50.420277 7254 raft_consensus.cc:3037] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:50.421568 6973 raft_consensus.cc:2454] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46d4901973984f6fbb1df5210ef8119c in term 1.
I20260808 10:39:50.421568 7254 raft_consensus.cc:2454] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46d4901973984f6fbb1df5210ef8119c in term 1.
I20260808 10:39:50.421883 7043 leader_election.cc:302] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fbc20d8fd5450b8553477b02d3df2b, 46d4901973984f6fbb1df5210ef8119c; no voters:
I20260808 10:39:50.422114 7335 raft_consensus.cc:2781] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:50.422369 7335 raft_consensus.cc:686] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 1 LEADER]: Becoming Leader. State: Replica: 46d4901973984f6fbb1df5210ef8119c, State: Running, Role: LEADER
I20260808 10:39:50.422683 7335 consensus_queue.cc:237] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } }
I20260808 10:39:50.425756 6816 catalog_manager.cc:5495] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c reported cstate change: term changed from 0 to 1, leader changed from <none> to 46d4901973984f6fbb1df5210ef8119c (127.1.194.66). New cstate: current_term: 1 leader_uuid: "46d4901973984f6fbb1df5210ef8119c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d4901973984f6fbb1df5210ef8119c" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 43593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:50.454730 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.453059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.1.194.66:43593)
0808 10:39:50.453203 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.453207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.453222 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:39:50.453333 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.453335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.453400 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.453624 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.454378 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.454382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.454384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.454396 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.454416 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.454424 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.454508 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.454583 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.454621 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:39:50.461160 6973 raft_consensus.cc:1261] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 1 FOLLOWER]: Refusing update from remote peer 46d4901973984f6fbb1df5210ef8119c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:50.461161 7254 raft_consensus.cc:1261] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 1 FOLLOWER]: Refusing update from remote peer 46d4901973984f6fbb1df5210ef8119c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:50.461978 7335 consensus_queue.cc:1035] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fbc20d8fd5450b8553477b02d3df2b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 43179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:50.462180 7341 consensus_queue.cc:1035] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f8fbad4e1904423a280555c72660dbb" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:50.470134 7346 mvcc.cc:205] Tried to move back new op lower bound from 7316216178520694784 to 7316216178373476352. Current Snapshot: MvccSnapshot[applied={T|T < 7316216178520694784}]
I20260808 10:39:50.477885 6816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:50.478058 6816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:50.480458 6816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64b7010ac6a44ef4b3704efa890e7fff: Sending DeleteTablet for 3 replicas of tablet 80c94ceb5ddd4e5d8132a9c200e36003
I20260808 10:39:50.481954 6951 tablet_service.cc:1514] Processing DeleteTablet for tablet 80c94ceb5ddd4e5d8132a9c200e36003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:50 UTC) from {username='slave'} at 127.0.0.1:49078
I20260808 10:39:50.482602 7355 tablet_replica.cc:330] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b: stopping tablet replica
I20260808 10:39:50.483011 7355 raft_consensus.cc:2229] T 80c94ceb5ddd4e5d8132a9c200e36003 P 33fbc20d8fd5450b8553477b02d3df2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:50.483251 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6884
I20260808 10:39:50.486183 7231 tablet_service.cc:1514] Processing DeleteTablet for tablet 80c94ceb5ddd4e5d8132a9c200e36003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:50 UTC) from {username='slave'} at 127.0.0.1:43262
I20260808 10:39:50.486783 7357 tablet_replica.cc:330] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: stopping tablet replica
I20260808 10:39:50.487210 7357 raft_consensus.cc:2229] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:50.487633 7357 raft_consensus.cc:2258] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:50.489688 7357 ts_tablet_manager.cc:1905] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:50.493844 7357 ts_tablet_manager.cc:1918] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:50.493942 7357 log.cc:1196] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/80c94ceb5ddd4e5d8132a9c200e36003
I20260808 10:39:50.494246 7357 ts_tablet_manager.cc:1939] T 80c94ceb5ddd4e5d8132a9c200e36003 P 4f8fbad4e1904423a280555c72660dbb: Deleting consensus metadata
I20260808 10:39:50.495621 7089 tablet_service.cc:1514] Processing DeleteTablet for tablet 80c94ceb5ddd4e5d8132a9c200e36003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:50 UTC) from {username='slave'} at 127.0.0.1:37778
I20260808 10:39:50.496176 7358 tablet_replica.cc:330] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c: stopping tablet replica
I20260808 10:39:50.496193 6798 catalog_manager.cc:4841] TS 4f8fbad4e1904423a280555c72660dbb (127.1.194.67:36163): tablet 80c94ceb5ddd4e5d8132a9c200e36003 (table test-table [id=415afb52bb7d4f70996a1e7c6aab032f]) successfully deleted
I20260808 10:39:50.496537 7358 raft_consensus.cc:2229] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:50.502226 7358 raft_consensus.cc:2258] T 80c94ceb5ddd4e5d8132a9c200e36003 P 46d4901973984f6fbb1df5210ef8119c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:39:50.502601 6800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:43179 (error 108)
I20260808 10:39:50.502859 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7025
I20260808 10:39:50.506114 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.505719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:43179 (local address 127.0.0.1:49094)
0808 10:39:50.505903 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.506001 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:43179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 10:39:50.506318 6800 catalog_manager.cc:4584] TS 33fbc20d8fd5450b8553477b02d3df2b (127.1.194.65:43179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 80c94ceb5ddd4e5d8132a9c200e36003: Network error: Client connection negotiation failed: client connection to 127.1.194.65:43179: connect: Connection refused (error 111)
I20260808 10:39:50.510416 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7164
I20260808 10:39:50.514923 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.514634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:43593 (local address 127.0.0.1:37786)
0808 10:39:50.514782 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.514846 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.66:43593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 10:39:50.519090 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 6782
2026-08-08T10:39:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1054 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 10:39:50.542665 1801 mini_oidc.cc:152] Starting JWKS server
I20260808 10:39:50.544616 1801 webserver.cc:460] Webserver started at https://127.0.0.1:35729/ using document root <none> and password file <none>
I20260808 10:39:50.544813 1801 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:39:50.545147 1801 webserver.cc:460] Webserver started at http://127.0.0.1:38421/ using document root <none> and password file <none>
2026-08-08T10:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:50Z Disabled control of system clock
I20260808 10:39:50.556283 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:38085
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:38085
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35729/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/testchainca.cert with env {}
W20260808 10:39:50.668092 7365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:50.668350 7365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:50.668390 7365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:50.672432 7365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:50.672493 7365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:50.672516 7365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:50.672571 7365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:50.672605 7365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:39:50.672624 7365 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:39:50.672650 7365 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35729/jwks.json
I20260808 10:39:50.677999 7365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:38085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:38085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35729/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:50.679066 7365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:50.680104 7365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:50.688551 7372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:50.688565 7371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:50.688661 7365 server_base.cc:1034] running on GCE node
W20260808 10:39:50.688570 7374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:50.689443 7365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:50.690667 7365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:50.691855 7365 hybrid_clock.cc:648] HybridClock initialized: now 1786185590691832 us; error 38 us; skew 500 ppm
I20260808 10:39:50.693892 7365 webserver.cc:460] Webserver started at http://127.1.194.126:33423/ using document root <none> and password file <none>
I20260808 10:39:50.694591 7365 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:50.694670 7365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:50.694972 7365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:50.697314 7365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "50ce736c42394ce3b376ab575ee27fb0"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "57076c321332d8cd8dcd318c7f47243b"
server_key_iv: "9ee584a9c536165c8855d72b1908e9ff"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.698017 7365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "50ce736c42394ce3b376ab575ee27fb0"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "57076c321332d8cd8dcd318c7f47243b"
server_key_iv: "9ee584a9c536165c8855d72b1908e9ff"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.702024 7365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:50.704597 7380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:50.706430 7365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:50.706555 7365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "50ce736c42394ce3b376ab575ee27fb0"
format_stamp: "Formatted at 2026-08-08 10:39:50 on dist-test-slave-ss4n"
server_key: "57076c321332d8cd8dcd318c7f47243b"
server_key_iv: "9ee584a9c536165c8855d72b1908e9ff"
server_key_version: "encryptionkey@0"
I20260808 10:39:50.706710 7365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:50.746317 7365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:50.747117 7365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:50.747345 7365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:50.755424 7365 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:38085
I20260808 10:39:50.755432 7441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:38085 every 8 connection(s)
I20260808 10:39:50.756453 7365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:50.759938 7442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:50.763265 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7365
I20260808 10:39:50.763382 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:50.763706 1801 external_mini_cluster.cc:1428] Setting key 7d2d46183918f2e7a7e71ba6556d0e11
I20260808 10:39:50.766222 7442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0: Bootstrap starting.
I20260808 10:39:50.768561 7442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:50.769408 7442 log.cc:824] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:50.771772 7442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0: No bootstrap required, opened a new log
I20260808 10:39:50.773947 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.765210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.1.194.126:38085)
0808 10:39:50.765967 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.765986 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.766027 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:39:50.766805 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.766824 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.767260 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.767576 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:50.767938 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:50.769048 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:50.770245 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:50.770842 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:50.770978 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:50.771938 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:50.771943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:50.771957 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:50.772370 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:50.772434 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:50.772455 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:50.772539 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:50.772751 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:39:50.773087 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":124,"threads_started":1}
I20260808 10:39:50.775733 7442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ce736c42394ce3b376ab575ee27fb0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38085 } }
I20260808 10:39:50.776002 7442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:50.776083 7442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ce736c42394ce3b376ab575ee27fb0, State: Initialized, Role: FOLLOWER
I20260808 10:39:50.776553 7442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ce736c42394ce3b376ab575ee27fb0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38085 } }
I20260808 10:39:50.776710 7442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:50.776777 7442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:50.776885 7442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:50.777007 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:50.778064 7442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ce736c42394ce3b376ab575ee27fb0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38085 } }
I20260808 10:39:50.778434 7442 leader_election.cc:302] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50ce736c42394ce3b376ab575ee27fb0; no voters:
I20260808 10:39:50.778784 7442 leader_election.cc:288] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:50.778977 7447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:50.779228 7447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [term 1 LEADER]: Becoming Leader. State: Replica: 50ce736c42394ce3b376ab575ee27fb0, State: Running, Role: LEADER
I20260808 10:39:50.779556 7447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ce736c42394ce3b376ab575ee27fb0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38085 } }
I20260808 10:39:50.780025 7442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:50.781298 7448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50ce736c42394ce3b376ab575ee27fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ce736c42394ce3b376ab575ee27fb0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38085 } } }
I20260808 10:39:50.781468 7448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:50.782676 7449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50ce736c42394ce3b376ab575ee27fb0. Latest consensus state: current_term: 1 leader_uuid: "50ce736c42394ce3b376ab575ee27fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ce736c42394ce3b376ab575ee27fb0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38085 } } }
I20260808 10:39:50.782789 7449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:50.782826 7458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:50.785838 7458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:50.791715 7458 catalog_manager.cc:1347] Generated new cluster ID: 22b218da61eb433e932f4f21b083b142
I20260808 10:39:50.791787 7458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:50.796815 7458 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:50.798103 7458 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:50.807577 7458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50ce736c42394ce3b376ab575ee27fb0: Generated new TSK 0
I20260808 10:39:50.808389 7458 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:39:50.837218 7473 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 10:39:50.837387 7473 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:39:50.837962 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:50.819743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.1.194.126:38085)
0808 10:39:50.819887 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:50.819893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:50.820490 (+ 597us) server_negotiation.cc:407] Connection header received
0808 10:39:50.820710 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:50.820715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:50.820778 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:50.820870 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:39:50.821223 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:50.822084 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:50.823035 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:50.823387 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:50.823490 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:50.823793 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:39:50.837552 (+ 13759us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:39:50.837814 (+ 262us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44730: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 10:39:50.838027 7445 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44730: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 10:39:50.839228 7472 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:50.819522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:38085 (local address 127.0.0.1:44730)
0808 10:39:50.820362 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:50.820379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:50.820525 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:50.820929 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:50.820933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:50.820945 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:39:50.821102 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:50.821111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:50.822229 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:50.822232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:50.822910 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:50.822920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:50.823497 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:50.823500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:50.823572 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:50.823612 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:39:50.837768 (+ 14156us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:39:50.837956 (+ 188us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.194.126:38085: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":134,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.194.126:38085: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 10:39:50.840252 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7365
2026-08-08T10:39:50Z chronyd exiting
I20260808 10:39:50.879252 1801 test_util.cc:175] -----------------------------------------------
I20260808 10:39:50.879369 1801 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185564023815-1801-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 10:39:50.880565 1801 mini_oidc.cc:152] Starting JWKS server
I20260808 10:39:50.881523 1801 webserver.cc:460] Webserver started at https://127.0.0.1:38457/ using document root <none> and password file <none>
I20260808 10:39:50.881665 1801 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:39:50.881927 1801 webserver.cc:460] Webserver started at http://127.0.0.1:43213/ using document root <none> and password file <none>
2026-08-08T10:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:50Z Disabled control of system clock
I20260808 10:39:50.892597 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:35205
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:35205
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38457/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/testchainca.cert with env {}
W20260808 10:39:51.012615 7479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:51.012874 7479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:51.012915 7479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:51.016904 7479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:51.016966 7479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:51.016983 7479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:51.016999 7479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:51.017015 7479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:39:51.017030 7479 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:39:51.017050 7479 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38457/jwks.json
I20260808 10:39:51.022586 7479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:35205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:35205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38457/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:51.023556 7479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:51.024488 7479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:39:51.033012 7479 server_base.cc:1034] running on GCE node
W20260808 10:39:51.033006 7485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:51.033038 7486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:51.033001 7489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:51.033885 7479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:51.034996 7479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:51.036197 7479 hybrid_clock.cc:648] HybridClock initialized: now 1786185591036171 us; error 44 us; skew 500 ppm
I20260808 10:39:51.043754 7479 webserver.cc:460] Webserver started at http://127.1.194.126:35965/ using document root <none> and password file <none>
I20260808 10:39:51.044414 7479 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:51.044518 7479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:51.044823 7479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:51.046996 7479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "8c9479d386a34d8abde159bec21dcc75"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "27d6b55765f7607b942bafbd6f562093"
server_key_iv: "a46c5ccb3ce5978f176bcc37f246cd96"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.047571 7479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "8c9479d386a34d8abde159bec21dcc75"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "27d6b55765f7607b942bafbd6f562093"
server_key_iv: "a46c5ccb3ce5978f176bcc37f246cd96"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.051403 7479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:51.053874 7495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:51.055135 7479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 10:39:51.055281 7479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "8c9479d386a34d8abde159bec21dcc75"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "27d6b55765f7607b942bafbd6f562093"
server_key_iv: "a46c5ccb3ce5978f176bcc37f246cd96"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.055399 7479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:51.070638 7479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:51.071376 7479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:51.071588 7479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:51.078766 7479 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:35205
I20260808 10:39:51.078820 7552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:35205 every 8 connection(s)
I20260808 10:39:51.079779 7479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:51.082967 7553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:51.088294 7553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75: Bootstrap starting.
I20260808 10:39:51.089244 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7479
I20260808 10:39:51.089349 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:51.089738 1801 external_mini_cluster.cc:1428] Setting key 0dfc9f7d4fdd4a51be018597457c0ab9
I20260808 10:39:51.091177 7553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:51.091980 7553 log.cc:824] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:51.094079 7553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75: No bootstrap required, opened a new log
I20260808 10:39:51.098397 7553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9479d386a34d8abde159bec21dcc75" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35205 } }
I20260808 10:39:51.098625 7553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:51.098685 7553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9479d386a34d8abde159bec21dcc75, State: Initialized, Role: FOLLOWER
I20260808 10:39:51.098762 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:51.090816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.1.194.126:35205)
0808 10:39:51.091490 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:51.091504 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:51.091544 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:39:51.092234 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:51.092252 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:51.092634 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:51.092903 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:51.093232 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.094293 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.095471 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.095917 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.096036 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.096923 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:51.096927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:51.096939 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:51.097320 (+ 381us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:51.097373 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:51.097393 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:51.097468 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:51.097676 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:39:51.097979 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":135,"threads_started":1}
I20260808 10:39:51.099141 7553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9479d386a34d8abde159bec21dcc75" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35205 } }
I20260808 10:39:51.099293 7553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:51.099380 7553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:51.099478 7553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:51.100620 7553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9479d386a34d8abde159bec21dcc75" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35205 } }
I20260808 10:39:51.100970 7553 leader_election.cc:302] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c9479d386a34d8abde159bec21dcc75; no voters:
I20260808 10:39:51.101351 7553 leader_election.cc:288] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:51.101444 7559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:51.101688 7559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9479d386a34d8abde159bec21dcc75, State: Running, Role: LEADER
I20260808 10:39:51.101944 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:51.102030 7559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9479d386a34d8abde159bec21dcc75" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35205 } }
I20260808 10:39:51.102677 7553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:51.103925 7560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c9479d386a34d8abde159bec21dcc75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9479d386a34d8abde159bec21dcc75" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35205 } } }
I20260808 10:39:51.103917 7561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c9479d386a34d8abde159bec21dcc75. Latest consensus state: current_term: 1 leader_uuid: "8c9479d386a34d8abde159bec21dcc75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9479d386a34d8abde159bec21dcc75" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35205 } } }
I20260808 10:39:51.104135 7560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:51.104167 7561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75 [sys.catalog]: This master's current role is: LEADER
W20260808 10:39:51.107390 7575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:51.107493 7575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:51.107613 7569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:51.108531 7569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:51.113651 7569 catalog_manager.cc:1347] Generated new cluster ID: 3788b04c5c8344929edf34dac9ff0a9d
I20260808 10:39:51.113718 7569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:51.126431 7569 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:51.127363 7569 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:51.132959 7569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c9479d386a34d8abde159bec21dcc75: Generated new TSK 0
I20260808 10:39:51.133762 7569 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:39:51.166844 7585 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 10:39:51.166947 7585 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:39:51.167659 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:51.150991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.1.194.126:35205)
0808 10:39:51.151132 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:51.151137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:51.151297 (+ 160us) server_negotiation.cc:407] Connection header received
0808 10:39:51.151434 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:51.151437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:51.151497 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:51.151622 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:39:51.151824 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.152667 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.153618 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.153971 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.154086 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.154365 (+ 279us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:39:51.167245 (+ 12880us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:39:51.167512 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56186: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 10:39:51.167730 7557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56186: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 10:39:51.167812 7584 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:51.150901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:35205 (local address 127.0.0.1:56186)
0808 10:39:51.151211 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:51.151224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:51.151319 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:51.151623 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:51.151626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:51.151632 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:39:51.151721 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:51.151726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.152789 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.152792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:51.153451 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:51.153459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.154091 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.154094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:51.154165 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.154188 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:39:51.167437 (+ 13249us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:39:51.167622 (+ 185us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.194.126:35205: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":132,"threads_started":1}
I20260808 10:39:51.168470 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7479
2026-08-08T10:39:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (328 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 10:39:51.209441 1801 mini_oidc.cc:152] Starting JWKS server
I20260808 10:39:51.210583 1801 webserver.cc:460] Webserver started at https://127.0.0.1:35515/ using document root <none> and password file <none>
I20260808 10:39:51.210752 1801 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:39:51.211083 1801 webserver.cc:460] Webserver started at http://127.0.0.1:35957/ using document root <none> and password file <none>
2026-08-08T10:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:51Z Disabled control of system clock
I20260808 10:39:51.222296 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:44773
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:44773
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35515/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:39:51.339277 7592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:51.339560 7592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:51.339599 7592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:51.343698 7592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:51.343770 7592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:51.343786 7592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:51.343802 7592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:51.343818 7592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:39:51.343834 7592 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:39:51.343847 7592 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35515/jwks.json
I20260808 10:39:51.348656 7592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:44773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35515/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:51.349777 7592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:51.351172 7592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:51.359903 7598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:51.359848 7597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:51.359874 7600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:51.360296 7592 server_base.cc:1034] running on GCE node
I20260808 10:39:51.360858 7592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:51.362092 7592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:51.363313 7592 hybrid_clock.cc:648] HybridClock initialized: now 1786185591363298 us; error 31 us; skew 500 ppm
I20260808 10:39:51.365262 7592 webserver.cc:460] Webserver started at http://127.1.194.126:42521/ using document root <none> and password file <none>
I20260808 10:39:51.365911 7592 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:51.366008 7592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:51.366256 7592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:51.368333 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "c7fb701ec7ec4b6bbe0957c76073a394"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "e677e37bb169ab2965a7405ee1f9ff0d"
server_key_iv: "fd907f62a08ad12c08227b8e28fe86c0"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.368911 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "c7fb701ec7ec4b6bbe0957c76073a394"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "e677e37bb169ab2965a7405ee1f9ff0d"
server_key_iv: "fd907f62a08ad12c08227b8e28fe86c0"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.372776 7592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:39:51.375290 7606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:51.376503 7592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:39:51.376646 7592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "c7fb701ec7ec4b6bbe0957c76073a394"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "e677e37bb169ab2965a7405ee1f9ff0d"
server_key_iv: "fd907f62a08ad12c08227b8e28fe86c0"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.376773 7592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:51.398613 7592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:51.399324 7592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:51.399549 7592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:51.406785 7592 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:44773
I20260808 10:39:51.406801 7662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:44773 every 8 connection(s)
I20260808 10:39:51.407712 7592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:51.409593 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7592
I20260808 10:39:51.409719 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:51.410071 1801 external_mini_cluster.cc:1428] Setting key cc5dc9519b4381034f8d6a74cbd3d527
I20260808 10:39:51.411602 7663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:51.419483 7663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394: Bootstrap starting.
I20260808 10:39:51.420326 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:51.411441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.1.194.126:44773)
0808 10:39:51.412030 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:51.412046 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:51.412080 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:39:51.413038 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:51.413061 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:51.413604 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:51.413908 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:51.414248 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.415225 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.416778 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.417264 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.417389 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.418418 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:51.418422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:51.418434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:51.418882 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:51.418938 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:51.418959 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:51.419038 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:51.419216 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 10:39:51.419506 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":129,"threads_started":1}
I20260808 10:39:51.422012 7663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:51.422940 7663 log.cc:824] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:51.423319 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:51.425156 7663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394: No bootstrap required, opened a new log
I20260808 10:39:51.428771 7663 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44773 } }
I20260808 10:39:51.429020 7663 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:51.429106 7663 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7fb701ec7ec4b6bbe0957c76073a394, State: Initialized, Role: FOLLOWER
I20260808 10:39:51.429644 7663 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44773 } }
I20260808 10:39:51.429801 7663 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:51.429898 7663 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:51.430022 7663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:51.431001 7663 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44773 } }
I20260808 10:39:51.431365 7663 leader_election.cc:302] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7fb701ec7ec4b6bbe0957c76073a394; no voters:
I20260808 10:39:51.431711 7663 leader_election.cc:288] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:51.431922 7669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:51.432186 7669 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [term 1 LEADER]: Becoming Leader. State: Replica: c7fb701ec7ec4b6bbe0957c76073a394, State: Running, Role: LEADER
I20260808 10:39:51.432483 7669 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44773 } }
I20260808 10:39:51.433029 7663 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:51.434365 7671 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7fb701ec7ec4b6bbe0957c76073a394. Latest consensus state: current_term: 1 leader_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44773 } } }
I20260808 10:39:51.434345 7670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fb701ec7ec4b6bbe0957c76073a394" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 44773 } } }
I20260808 10:39:51.434536 7670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:51.434800 7671 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:51.434996 7678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:51.438511 7678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:51.443708 7678 catalog_manager.cc:1347] Generated new cluster ID: 20d75f3b249c40c09f45dbf4769b9645
I20260808 10:39:51.443792 7678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:51.464347 7678 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:51.465425 7678 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:51.470108 7678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7fb701ec7ec4b6bbe0957c76073a394: Generated new TSK 0
I20260808 10:39:51.471128 7678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:51.494369 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:51.489110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59916 (local address 127.1.194.126:44773)
0808 10:39:51.489278 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:51.489285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:51.489458 (+ 173us) server_negotiation.cc:407] Connection header received
0808 10:39:51.489615 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:51.489618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:51.489670 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:51.489777 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:39:51.490047 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.490642 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.491879 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.492169 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.492305 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.492602 (+ 297us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:39:51.493976 (+ 1374us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35515/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:39:51.494254 (+ 278us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35515/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 10:39:51.494467 7666 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35515/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 10:39:51.497273 7696 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:51.489002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:44773 (local address 127.0.0.1:59916)
0808 10:39:51.489333 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:51.489348 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:51.489470 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:51.489796 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:51.489800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:51.489808 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:39:51.489930 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:51.489936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.490762 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.490766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:51.491760 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:51.491770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.492312 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.492316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:51.492404 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.492431 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:39:51.494153 (+ 1722us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35515/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:39:51.497097 (+ 2944us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.194.126:44773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35515/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":2730,"thread_start_us":122,"threads_started":1}
I20260808 10:39:51.498047 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7592
E20260808 10:39:51.507918 7697 webserver.cc:533] Webserver: sslize failed:
E20260808 10:39:51.508004 7697 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T10:39:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (330 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T10:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:51Z Disabled control of system clock
I20260808 10:39:51.549425 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:38247
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:38247
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/testchainca.cert with env {}
W20260808 10:39:51.660614 7701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:51.660878 7701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:51.660917 7701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:51.664915 7701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:51.664973 7701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:51.664989 7701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:51.665006 7701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:51.665028 7701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:39:51.665045 7701 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:39:51.665058 7701 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 10:39:51.670270 7701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:38247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:51.671247 7701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:51.672209 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:51.681185 7710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:51.681211 7707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:51.681146 7708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:51.681550 7701 server_base.cc:1034] running on GCE node
I20260808 10:39:51.682049 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:51.683183 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:51.684353 7701 hybrid_clock.cc:648] HybridClock initialized: now 1786185591684327 us; error 40 us; skew 500 ppm
I20260808 10:39:51.686278 7701 webserver.cc:460] Webserver started at http://127.1.194.126:45039/ using document root <none> and password file <none>
I20260808 10:39:51.686847 7701 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:51.686910 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:51.687153 7701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:51.689165 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "058f5d0cf1fa452a8a33dee9f5a287a0"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "6499b9d04ea8d855f4f7d025354b3fa7"
server_key_iv: "4707dfb734d0718c42b4cef63da8e8ef"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.689740 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "058f5d0cf1fa452a8a33dee9f5a287a0"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "6499b9d04ea8d855f4f7d025354b3fa7"
server_key_iv: "4707dfb734d0718c42b4cef63da8e8ef"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.693425 7701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:39:51.696058 7716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:51.697342 7701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:39:51.697542 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "058f5d0cf1fa452a8a33dee9f5a287a0"
format_stamp: "Formatted at 2026-08-08 10:39:51 on dist-test-slave-ss4n"
server_key: "6499b9d04ea8d855f4f7d025354b3fa7"
server_key_iv: "4707dfb734d0718c42b4cef63da8e8ef"
server_key_version: "encryptionkey@0"
I20260808 10:39:51.697676 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:51.723519 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:51.724292 7701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:51.724524 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:51.732205 7701 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:38247
I20260808 10:39:51.732214 7775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:38247 every 8 connection(s)
I20260808 10:39:51.733217 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:51.735817 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7701
I20260808 10:39:51.735993 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:51.736339 1801 external_mini_cluster.cc:1428] Setting key 4eb393fa6482f27fdeddfa0f1f61158d
I20260808 10:39:51.736811 7776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:51.743655 7776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0: Bootstrap starting.
I20260808 10:39:51.746052 7776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:51.746822 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:51.737851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.1.194.126:38247)
0808 10:39:51.738620 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:51.738637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:51.738675 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:39:51.739665 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:51.739685 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:51.740130 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:51.740600 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:51.741065 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.742242 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.743226 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.743769 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.743947 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.744910 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:51.744920 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:51.744934 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:51.745346 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:51.745403 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:51.745431 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:51.745608 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:51.745777 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:39:51.746054 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":163,"threads_started":1}
I20260808 10:39:51.746845 7776 log.cc:824] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:51.748881 7776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0: No bootstrap required, opened a new log
I20260808 10:39:51.749961 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:51.752840 7776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38247 } }
I20260808 10:39:51.753083 7776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:51.753155 7776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058f5d0cf1fa452a8a33dee9f5a287a0, State: Initialized, Role: FOLLOWER
I20260808 10:39:51.753674 7776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38247 } }
I20260808 10:39:51.753814 7776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:51.753906 7776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:51.754031 7776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:51.755218 7776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38247 } }
I20260808 10:39:51.755584 7776 leader_election.cc:302] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 058f5d0cf1fa452a8a33dee9f5a287a0; no voters:
I20260808 10:39:51.755908 7776 leader_election.cc:288] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:51.756042 7781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:51.756297 7781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [term 1 LEADER]: Becoming Leader. State: Replica: 058f5d0cf1fa452a8a33dee9f5a287a0, State: Running, Role: LEADER
I20260808 10:39:51.756590 7781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38247 } }
I20260808 10:39:51.757175 7776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:51.758379 7783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 058f5d0cf1fa452a8a33dee9f5a287a0. Latest consensus state: current_term: 1 leader_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38247 } } }
I20260808 10:39:51.758525 7783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:51.758368 7782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058f5d0cf1fa452a8a33dee9f5a287a0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 38247 } } }
I20260808 10:39:51.758771 7782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:51.759470 7790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:51.762782 7790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:51.768427 7790 catalog_manager.cc:1347] Generated new cluster ID: edf46ae45a7046ac8dcf7d16d74d77ce
I20260808 10:39:51.768499 7790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:51.781426 7790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:51.782789 7790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:51.789618 7790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 058f5d0cf1fa452a8a33dee9f5a287a0: Generated new TSK 0
I20260808 10:39:51.790273 7790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:52.277015 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:51.800191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.1.194.126:38247)
0808 10:39:51.800332 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:51.800337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:51.800512 (+ 175us) server_negotiation.cc:407] Connection header received
0808 10:39:51.800666 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:51.800669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:51.800723 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:51.800828 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:39:51.801049 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.801899 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.802989 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.803329 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.803432 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.803706 (+ 274us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:39:52.276597 (+472891us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:39:52.276884 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44980: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 10:39:52.277154 7779 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44980: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 10:39:52.277323 7805 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:51.800084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:38247 (local address 127.0.0.1:44980)
0808 10:39:51.800411 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:51.800423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:51.800527 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:51.800832 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:51.800835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:51.800840 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:39:51.800944 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:51.800949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.802082 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.802088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:51.802874 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:51.802884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:51.803441 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:51.803443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:51.803510 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:51.803533 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:39:52.276872 (+473339us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:39:52.277094 (+ 222us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.194.126:38247: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260808 10:39:52.278074 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7701
2026-08-08T10:39:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (760 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (144 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:52 dist-test-slave-ss4n krb5kdc[7817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:52 dist-test-slave-ss4n krb5kdc[7817](info): set up 2 sockets
Aug 08 10:39:52 dist-test-slave-ss4n krb5kdc[7817](info): commencing operation
krb5kdc: starting...
W20260808 10:39:53.656011 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:53 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:39:53.792627 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34407
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:53.903651 7832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:53.903975 7832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:53.904074 7832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:53.908269 7832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:53.908370 7832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:53.908412 7832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:53.908452 7832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:53.908483 7832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:53.914105 7832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:53.915702 7832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:53.917194 7832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:53.928223 7840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:53.928223 7841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:53.928766 7843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:53.928934 7832 server_base.cc:1034] running on GCE node
I20260808 10:39:53.929776 7832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:53.931349 7832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:53.932598 7832 hybrid_clock.cc:648] HybridClock initialized: now 1786185593932563 us; error 51 us; skew 500 ppm
Aug 08 10:39:53 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:53.936167 7832 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:39:53.937064 7832 webserver.cc:460] Webserver started at http://127.1.194.126:43509/ using document root <none> and password file <none>
I20260808 10:39:53.937808 7832 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:53.937915 7832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:53.938172 7832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:53.941092 7832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "cafbffcda0024a5597ee1d558af4041f"
format_stamp: "Formatted at 2026-08-08 10:39:53 on dist-test-slave-ss4n"
server_key: "6bd9183b90b2cb327db04eb7e8e4f595"
server_key_iv: "5688950043797aad7756ed6ce4c821e7"
server_key_version: "encryptionkey@0"
I20260808 10:39:53.941881 7832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "cafbffcda0024a5597ee1d558af4041f"
format_stamp: "Formatted at 2026-08-08 10:39:53 on dist-test-slave-ss4n"
server_key: "6bd9183b90b2cb327db04eb7e8e4f595"
server_key_iv: "5688950043797aad7756ed6ce4c821e7"
server_key_version: "encryptionkey@0"
I20260808 10:39:53.946020 7832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:39:53.948552 7852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:53.950054 7832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:53.950201 7832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "cafbffcda0024a5597ee1d558af4041f"
format_stamp: "Formatted at 2026-08-08 10:39:53 on dist-test-slave-ss4n"
server_key: "6bd9183b90b2cb327db04eb7e8e4f595"
server_key_iv: "5688950043797aad7756ed6ce4c821e7"
server_key_version: "encryptionkey@0"
I20260808 10:39:53.950320 7832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:53.962007 7832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:53.965418 7832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:53.965855 7832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:53.973446 7832 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34407
I20260808 10:39:53.973577 7906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34407 every 8 connection(s)
I20260808 10:39:53.974661 7832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:53.977635 7907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:53.979422 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7832
I20260808 10:39:53.979552 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:53.979849 1801 external_mini_cluster.cc:1428] Setting key 41f33211ba98e118579a649dc2cedfbf
I20260808 10:39:53.984157 7907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f: Bootstrap starting.
I20260808 10:39:53.986513 7907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:53.987282 7907 log.cc:824] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:39:53 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:53.989280 7907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f: No bootstrap required, opened a new log
I20260808 10:39:53.994344 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:53.981273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.1.194.126:34407)
0808 10:39:53.981717 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:53.981732 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:53.981767 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:39:53.982428 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:53.982446 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:53.982759 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:53.983097 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:53.983883 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:53.984907 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:53.985819 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:53.986283 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:53.986467 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:53.988994 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:53.988998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:53.989011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:53.989036 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:53.991158 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:53.991618 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:53.991624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:53.991630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:53.991803 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:53.992096 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:53.992102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:53.992104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:53.992648 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:53.992850 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:53.993053 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 10:39:53.993349 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":149,"threads_started":1}
I20260808 10:39:53.994308 7907 raft_consensus.cc:348] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafbffcda0024a5597ee1d558af4041f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34407 } }
I20260808 10:39:53.994598 7907 raft_consensus.cc:374] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:53.994665 7907 raft_consensus.cc:729] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafbffcda0024a5597ee1d558af4041f, State: Initialized, Role: FOLLOWER
I20260808 10:39:53.995141 7907 consensus_queue.cc:260] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafbffcda0024a5597ee1d558af4041f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34407 } }
I20260808 10:39:53.995306 7907 raft_consensus.cc:388] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:53.995380 7907 raft_consensus.cc:482] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:53.995487 7907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:53.996672 7907 raft_consensus.cc:504] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafbffcda0024a5597ee1d558af4041f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34407 } }
I20260808 10:39:53.997022 7907 leader_election.cc:302] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cafbffcda0024a5597ee1d558af4041f; no voters:
I20260808 10:39:53.997354 7907 leader_election.cc:288] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:53.997468 7914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:53.997766 7914 raft_consensus.cc:686] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [term 1 LEADER]: Becoming Leader. State: Replica: cafbffcda0024a5597ee1d558af4041f, State: Running, Role: LEADER
I20260808 10:39:53.998134 7914 consensus_queue.cc:237] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafbffcda0024a5597ee1d558af4041f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34407 } }
I20260808 10:39:53.998656 7907 sys_catalog.cc:564] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:53.999884 7916 sys_catalog.cc:455] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cafbffcda0024a5597ee1d558af4041f. Latest consensus state: current_term: 1 leader_uuid: "cafbffcda0024a5597ee1d558af4041f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafbffcda0024a5597ee1d558af4041f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34407 } } }
I20260808 10:39:53.999884 7915 sys_catalog.cc:455] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cafbffcda0024a5597ee1d558af4041f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafbffcda0024a5597ee1d558af4041f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34407 } } }
I20260808 10:39:54.000041 7916 sys_catalog.cc:458] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:54.000041 7915 sys_catalog.cc:458] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:54.000537 7925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:54.003731 7925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:54.009461 7925 catalog_manager.cc:1347] Generated new cluster ID: d3d03aa16ad440789355c19552c09e4e
I20260808 10:39:54.009593 7925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:54.029058 7925 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:54.030092 7925 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:54.040393 7925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cafbffcda0024a5597ee1d558af4041f: Generated new TSK 0
I20260808 10:39:54.041146 7925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:39:54.082928 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34407
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:54.223523 7937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:54.223788 7937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:54.223832 7937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:54.227725 7937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:54.227779 7937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:54.227835 7937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:54.232977 7937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:54.233925 7937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:54.234843 7937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:54.244230 7945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:54.244246 7944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:54.244230 7947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:54.244526 7937 server_base.cc:1034] running on GCE node
I20260808 10:39:54.244941 7937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:54.245551 7937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:54.246766 7937 hybrid_clock.cc:648] HybridClock initialized: now 1786185594246753 us; error 29 us; skew 500 ppm
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.250003 7937 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:39:54.251031 7937 webserver.cc:460] Webserver started at http://127.1.194.65:42469/ using document root <none> and password file <none>
I20260808 10:39:54.251744 7937 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:54.251837 7937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:54.252065 7937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:54.254451 7937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "8b44d26ebd7b462bbd5482812cf20929"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "1c7038a5128c64badb2d38f53e8e3a0b"
server_key_iv: "ee75c10a551394b7855f1c1ae5313011"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.255098 7937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "8b44d26ebd7b462bbd5482812cf20929"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "1c7038a5128c64badb2d38f53e8e3a0b"
server_key_iv: "ee75c10a551394b7855f1c1ae5313011"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.259238 7937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:54.262032 7954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.263463 7937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:54.263595 7937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "8b44d26ebd7b462bbd5482812cf20929"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "1c7038a5128c64badb2d38f53e8e3a0b"
server_key_iv: "ee75c10a551394b7855f1c1ae5313011"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.263726 7937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:54.278187 7937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:54.281545 7937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:54.281755 7937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:54.282364 7937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:54.283190 7937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:54.283272 7937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.283342 7937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:54.283380 7937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.292351 7937 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:35011
I20260808 10:39:54.292367 8077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:35011 every 8 connection(s)
I20260808 10:39:54.293238 7937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:54.300721 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 7937
I20260808 10:39:54.300868 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:54.301214 1801 external_mini_cluster.cc:1428] Setting key 365a128f38a64e90f10712df14a41021
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:54.308656 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.295000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:56773 (local address 127.1.194.126:34407)
0808 10:39:54.295159 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.295164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.296090 (+ 926us) server_negotiation.cc:407] Connection header received
0808 10:39:54.297541 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.297593 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.297645 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.297721 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.298894 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.299718 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.301080 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.301459 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.301597 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.304556 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:54.304562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:54.304568 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:54.304601 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:54.306284 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.306906 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.306908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.306910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.306981 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.307378 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.307380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.307381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.307549 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:54.307627 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:54.308385 (+ 758us) server_negotiation.cc:299] Negotiation successful
0808 10:39:54.308530 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:39:54.309849 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.295429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34407 (local address 127.1.194.65:56773)
0808 10:39:54.295923 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:54.295966 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:54.297285 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:54.297881 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:54.297888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:54.298236 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:54.298760 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:54.298774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.299834 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.299837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:54.300944 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:54.300958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.301677 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.301683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:54.301894 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.302542 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:54.302565 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:54.304382 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:54.306402 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:54.306411 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:54.306429 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:54.306808 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:54.307082 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:54.307088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:54.307093 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:54.307282 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:54.307679 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:54.307689 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:54.308041 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 10:39:54.308274 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 10:39:54.308650 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":114,"threads_started":1}
I20260808 10:39:54.311461 8078 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34407
I20260808 10:39:54.311738 8078 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:54.312337 8078 heartbeater.cc:511] Master 127.1.194.126:34407 requested a full tablet report, sending...
I20260808 10:39:54.314021 7870 ts_manager.cc:194] Registered new tserver with Master: 8b44d26ebd7b462bbd5482812cf20929 (127.1.194.65:35011)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:39:54.315476 7870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:56773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:39:54.334223 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34407
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:54.449281 8085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:54.449648 8085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:54.449703 8085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:54.454469 8085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:54.454536 8085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:54.454646 8085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:54.460675 8085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:54.461735 8085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:54.462742 8085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:54.472527 8094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:54.472666 8085 server_base.cc:1034] running on GCE node
W20260808 10:39:54.472573 8091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:54.472538 8092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:54.473187 8085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:54.473948 8085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:54.475139 8085 hybrid_clock.cc:648] HybridClock initialized: now 1786185594475106 us; error 45 us; skew 500 ppm
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.478252 8085 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:39:54.479215 8085 webserver.cc:460] Webserver started at http://127.1.194.66:42523/ using document root <none> and password file <none>
I20260808 10:39:54.479789 8085 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:54.479879 8085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:54.480123 8085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:54.482249 8085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "7d08b87b985841daac64838ab6e409b8"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "db8b8a1b7dde49d313280e0c972d947e"
server_key_iv: "62e6a3febebc48b65a51b4a10b16ed15"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.482810 8085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "7d08b87b985841daac64838ab6e409b8"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "db8b8a1b7dde49d313280e0c972d947e"
server_key_iv: "62e6a3febebc48b65a51b4a10b16ed15"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.486447 8085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:39:54.488950 8102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.490238 8085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:54.490377 8085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "7d08b87b985841daac64838ab6e409b8"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "db8b8a1b7dde49d313280e0c972d947e"
server_key_iv: "62e6a3febebc48b65a51b4a10b16ed15"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.490489 8085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:54.515143 8085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:54.518337 8085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:54.518566 8085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:54.519197 8085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:54.520058 8085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:54.520130 8085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.520205 8085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:54.520249 8085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.529524 8085 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:37125
I20260808 10:39:54.529557 8224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:37125 every 8 connection(s)
I20260808 10:39:54.530572 8085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:54.532228 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8085
I20260808 10:39:54.532357 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:54.532702 1801 external_mini_cluster.cc:1428] Setting key f1a1a03157f463f939022426bd07be54
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:54.545363 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.532416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:57053 (local address 127.1.194.126:34407)
0808 10:39:54.532562 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.532566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.533519 (+ 953us) server_negotiation.cc:407] Connection header received
0808 10:39:54.535000 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.535006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.535076 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.535170 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.536340 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.536876 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.538121 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.538363 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.538479 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.541227 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:54.541231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:54.541234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:54.541267 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:54.543023 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.543631 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.543634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.543635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.543764 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.544291 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.544294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.544296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.544490 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:54.544574 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:54.545110 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 10:39:54.545224 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:54.546499 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.532860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34407 (local address 127.1.194.66:57053)
0808 10:39:54.533332 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:54.533386 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:54.534787 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:54.535321 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:54.535327 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:54.535729 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:54.536193 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:54.536206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.537010 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.537019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:54.537995 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:54.538005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.538517 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.538521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:54.538693 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.539315 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:54.539331 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:54.541029 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:54.543141 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:54.543145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:54.543157 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:54.543528 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:54.543893 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:54.543899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:54.543904 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:54.544084 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:54.544602 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:54.544608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:54.544853 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 10:39:54.545019 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:39:54.545459 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":134,"threads_started":1}
I20260808 10:39:54.547900 8225 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34407
I20260808 10:39:54.548213 8225 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:54.548863 8225 heartbeater.cc:511] Master 127.1.194.126:34407 requested a full tablet report, sending...
I20260808 10:39:54.550072 7870 ts_manager.cc:194] Registered new tserver with Master: 7d08b87b985841daac64838ab6e409b8 (127.1.194.66:37125)
I20260808 10:39:54.550724 7870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:57053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:39:54.564751 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34407
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:54.682847 8230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:54.683152 8230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:54.683200 8230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:54.687889 8230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:54.687968 8230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:54.688031 8230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:54.694738 8230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:54.695900 8230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:54.696924 8230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:54.707662 8239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:54.707738 8237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:54.707662 8236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:54.707870 8230 server_base.cc:1034] running on GCE node
I20260808 10:39:54.708442 8230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:54.709105 8230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:54.710275 8230 hybrid_clock.cc:648] HybridClock initialized: now 1786185594710255 us; error 34 us; skew 500 ppm
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.713351 8230 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:39:54.714371 8230 webserver.cc:460] Webserver started at http://127.1.194.67:43833/ using document root <none> and password file <none>
I20260808 10:39:54.714989 8230 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:54.715054 8230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:54.715287 8230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:54.717381 8230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "4fa6caa11bd1491ba46a8ad6ca5b9863"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "b840ace70c26efd3e440f6e351235477"
server_key_iv: "86aa273e35bcb0fbca06e6f23b2a1752"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.718040 8230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "4fa6caa11bd1491ba46a8ad6ca5b9863"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "b840ace70c26efd3e440f6e351235477"
server_key_iv: "86aa273e35bcb0fbca06e6f23b2a1752"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.722082 8230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:54.724685 8247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.726050 8230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:54.726179 8230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "4fa6caa11bd1491ba46a8ad6ca5b9863"
format_stamp: "Formatted at 2026-08-08 10:39:54 on dist-test-slave-ss4n"
server_key: "b840ace70c26efd3e440f6e351235477"
server_key_iv: "86aa273e35bcb0fbca06e6f23b2a1752"
server_key_version: "encryptionkey@0"
I20260808 10:39:54.726291 8230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:54.739637 8230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:54.742736 8230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:54.742941 8230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:54.743517 8230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:54.744328 8230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:54.744374 8230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.744439 8230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:54.744482 8230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:54.753585 8230 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:40075
I20260808 10:39:54.753604 8364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:40075 every 8 connection(s)
I20260808 10:39:54.754571 8230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:54.762786 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8230
I20260808 10:39:54.762943 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:54.763312 1801 external_mini_cluster.cc:1428] Setting key 926a86cd260cc5f9ce6adcc97b097e5d
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:39:54.770593 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.756504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:55747 (local address 127.1.194.126:34407)
0808 10:39:54.756721 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.756724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.757660 (+ 936us) server_negotiation.cc:407] Connection header received
0808 10:39:54.759225 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.759228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.759294 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.759418 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.760716 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.761309 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.762980 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.763292 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.763438 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.766401 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:54.766404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:54.766406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:54.766431 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:54.768181 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.768890 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.768893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.768895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.768980 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.769443 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.769445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.769447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.769727 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:54.769813 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:54.770367 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 10:39:54.770470 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 10:39:54.771521 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.756879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34407 (local address 127.1.194.67:55747)
0808 10:39:54.757389 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:54.757440 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:54.758987 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:54.759634 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:54.759641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:54.760026 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:54.760544 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:54.760559 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.761451 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.761455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:54.762791 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:54.762806 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.763494 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.763499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:54.763720 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.764421 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:54.764440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:54.766242 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:54.768407 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:54.768412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:54.768425 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:54.768770 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:54.769139 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:54.769142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:54.769144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:54.769300 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:54.769834 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:54.769839 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:54.770129 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 10:39:54.770290 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 10:39:54.770571 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":115,"threads_started":1}
I20260808 10:39:54.772671 8365 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34407
I20260808 10:39:54.772918 8365 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:54.773411 8365 heartbeater.cc:511] Master 127.1.194.126:34407 requested a full tablet report, sending...
I20260808 10:39:54.774433 7870 ts_manager.cc:194] Registered new tserver with Master: 4fa6caa11bd1491ba46a8ad6ca5b9863 (127.1.194.67:40075)
I20260808 10:39:54.775002 7870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:55747
I20260808 10:39:54.777415 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/test-admin.keytab.
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.795859 1801 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 10:39:54.798992 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:54.797399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.1.194.65:35011)
0808 10:39:54.797733 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.797762 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.797786 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:54.797850 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.797855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.798055 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.798195 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.798790 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52570: BlockingRecv error: recv got EOF from 127.0.0.1:52570 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":143,"threads_started":1}
W20260808 10:39:54.799113 1801 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.800341 1801 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.801400 1801 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:54.811642 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.802445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.1.194.65:35011)
0808 10:39:54.802652 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.802659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.802699 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:39:54.802858 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.802861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.802912 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.803005 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.803572 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.804345 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.805185 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.805631 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.805749 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.808061 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:54.808077 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:54.808091 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:54.808114 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:54.809861 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.810315 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.810320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.810322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.810400 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.810681 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.810686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.810687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.810935 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:54.811154 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:54.811340 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 10:39:54.811500 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 10:39:54.815627 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:54.814608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.1.194.65:35011)
0808 10:39:54.814733 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.814736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.814748 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:54.814852 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.814854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.814889 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.814951 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.815476 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52586: BlockingRecv error: recv got EOF from 127.0.0.1:52586 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 10:39:54.815742 1801 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.816759 1801 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.817703 1801 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:54.826707 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.818538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52596 (local address 127.1.194.65:35011)
0808 10:39:54.818658 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.818661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.818803 (+ 142us) server_negotiation.cc:407] Connection header received
0808 10:39:54.818880 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.818882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.818915 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.818978 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.819432 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.819988 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.820784 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.820989 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.821083 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.823384 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:54.823387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:54.823389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:54.823410 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:54.824835 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.825394 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.825399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.825401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.825543 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.825841 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.825845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.825847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.826015 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:54.826184 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:54.826403 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:39:54.826535 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 10:39:54.829190 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:54.828342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.1.194.65:35011)
0808 10:39:54.828454 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.828458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.828542 (+ 84us) server_negotiation.cc:407] Connection header received
0808 10:39:54.828645 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.828647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.828685 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.828749 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.829069 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52612: BlockingRecv error: recv got EOF from 127.0.0.1:52612 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:39:54.829329 1801 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185564023815-1801-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.830425 1801 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:54.831413 1801 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[7817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:39:54.840581 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:54.832157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.1.194.65:35011)
0808 10:39:54.832282 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:54.832286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:54.832443 (+ 157us) server_negotiation.cc:407] Connection header received
0808 10:39:54.832615 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:54.832620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:54.832668 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:54.832759 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:54.833246 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.833902 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.834951 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:54.835282 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:54.835375 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:54.837927 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:54.837930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:54.837932 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:54.837953 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:54.839250 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.839633 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.839635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.839637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.839707 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:54.839930 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:54.839932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:54.839934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:54.840084 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:54.840181 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:54.840369 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 10:39:54.840459 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:39:54.841688 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7937
I20260808 10:39:54.849591 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8085
I20260808 10:39:54.855441 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8230
I20260808 10:39:54.862954 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 7832
2026-08-08T10:39:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2311 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[8388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[8388](info): set up 2 sockets
Aug 08 10:39:54 dist-test-slave-ss4n krb5kdc[8388](info): commencing operation
krb5kdc: starting...
W20260808 10:39:55.964399 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab.
Entry for principal oryx/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab.
I20260808 10:39:56.096833 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42389
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab
--principal=oryx/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:56.208045 8404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:56.208350 8404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:56.208424 8404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:56.212706 8404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:56.212796 8404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:56.212836 8404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:56.212875 8404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:56.212908 8404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:56.218174 8404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab
--principal=oryx/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:56.219236 8404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:56.220263 8404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:56.228844 8414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:56.228844 8411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:56.228966 8410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:56.229271 8404 server_base.cc:1034] running on GCE node
I20260808 10:39:56.229902 8404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:56.230963 8404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:56.232123 8404 hybrid_clock.cc:648] HybridClock initialized: now 1786185596232097 us; error 40 us; skew 500 ppm
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:56.239845 8404 init.cc:377] Logged in from keytab as oryx/127.1.194.126@KRBTEST.COM (short username oryx)
I20260808 10:39:56.240738 8404 webserver.cc:460] Webserver started at http://127.1.194.126:35327/ using document root <none> and password file <none>
I20260808 10:39:56.241278 8404 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:56.241339 8404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:56.241674 8404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:56.243717 8404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "643614d186614f9ea5572a49969ba2e9"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "d70950ef7e65feefb19f89448a2801ca"
server_key_iv: "64e6b19501f497a49864cafa3ea72eab"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.244280 8404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "643614d186614f9ea5572a49969ba2e9"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "d70950ef7e65feefb19f89448a2801ca"
server_key_iv: "64e6b19501f497a49864cafa3ea72eab"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.247953 8404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:39:56.250299 8421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.251485 8404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:56.251633 8404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "643614d186614f9ea5572a49969ba2e9"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "d70950ef7e65feefb19f89448a2801ca"
server_key_iv: "64e6b19501f497a49864cafa3ea72eab"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.251748 8404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:56.286324 8404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:56.289314 8404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:56.289602 8404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:56.296828 8404 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:42389
I20260808 10:39:56.297206 8477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:42389 every 8 connection(s)
I20260808 10:39:56.297859 8404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:56.300853 8478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:56.303711 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8404
I20260808 10:39:56.303866 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:39:56.304205 1801 external_mini_cluster.cc:1428] Setting key fd237ac5544fd4c59bb5a36ea0022be0
I20260808 10:39:56.307566 8478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Bootstrap starting.
I20260808 10:39:56.310348 8478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:56.311254 8478 log.cc:824] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:56.313829 8478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: No bootstrap required, opened a new log
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:39:56.318054 8478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } }
I20260808 10:39:56.318368 8478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:56.318485 8478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643614d186614f9ea5572a49969ba2e9, State: Initialized, Role: FOLLOWER
I20260808 10:39:56.319028 8478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } }
I20260808 10:39:56.319187 8478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:56.319249 8478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:56.319353 8478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:56.320559 8478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } }
I20260808 10:39:56.320909 8478 leader_election.cc:302] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 643614d186614f9ea5572a49969ba2e9; no voters:
I20260808 10:39:56.321157 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:56.305764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43584 (local address 127.1.194.126:42389)
0808 10:39:56.306228 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:56.306244 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:56.306281 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:39:56.306948 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:56.306967 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:56.307302 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:56.307602 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:56.308607 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.309803 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.311076 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.311757 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.311966 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:56.314939 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:56.314945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:56.314971 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:56.315003 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:56.317732 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:56.318203 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:56.318209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:56.318215 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:56.318358 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:56.318731 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:56.318735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:56.318736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:56.319266 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:56.319477 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:56.319821 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 10:39:56.320133 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":161,"threads_started":1}
I20260808 10:39:56.321278 8478 leader_election.cc:288] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:56.321403 8483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:56.321652 8483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 LEADER]: Becoming Leader. State: Replica: 643614d186614f9ea5572a49969ba2e9, State: Running, Role: LEADER
I20260808 10:39:56.322029 8483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } }
I20260808 10:39:56.322593 8478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:56.323266 8483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 643614d186614f9ea5572a49969ba2e9. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } }
I20260808 10:39:56.323455 8483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:56.323881 8492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:56.324425 8484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } }
I20260808 10:39:56.324602 8484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:56.327488 8492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:56.333544 8492 catalog_manager.cc:1347] Generated new cluster ID: ed16ecec505c4e54809e973b9d4b105b
I20260808 10:39:56.333612 8492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:56.341110 8492 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:56.342120 8492 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:56.347292 8492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Generated new TSK 0
I20260808 10:39:56.347971 8492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab.
Entry for principal oryx/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab.
I20260808 10:39:56.389734 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42389
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:56.517458 8505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:56.517802 8505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:56.517848 8505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:56.521782 8505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:56.521833 8505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:56.521890 8505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:56.527187 8505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:56.528227 8505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:56.529175 8505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:56.538728 8510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:56.538765 8513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:56.539024 8505 server_base.cc:1034] running on GCE node
W20260808 10:39:56.538789 8511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:56.539511 8505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:56.540172 8505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:56.541399 8505 hybrid_clock.cc:648] HybridClock initialized: now 1786185596541350 us; error 70 us; skew 500 ppm
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:56.544159 8505 init.cc:377] Logged in from keytab as oryx/127.1.194.65@KRBTEST.COM (short username oryx)
I20260808 10:39:56.544958 8505 webserver.cc:460] Webserver started at http://127.1.194.65:40051/ using document root <none> and password file <none>
I20260808 10:39:56.545473 8505 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:56.545606 8505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:56.545778 8505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:56.547744 8505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "90ffd8c5865d476eb0c12d3ebc1895da"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "9f6103eab36ebadf9d14825ce3288220"
server_key_iv: "47e470fc179bc54596c71d74cd84f1d2"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.548254 8505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "90ffd8c5865d476eb0c12d3ebc1895da"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "9f6103eab36ebadf9d14825ce3288220"
server_key_iv: "47e470fc179bc54596c71d74cd84f1d2"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.551896 8505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:39:56.554281 8522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.555608 8505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:56.555704 8505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "90ffd8c5865d476eb0c12d3ebc1895da"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "9f6103eab36ebadf9d14825ce3288220"
server_key_iv: "47e470fc179bc54596c71d74cd84f1d2"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.555841 8505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:56.566295 8505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:56.569305 8505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:56.569459 8505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:56.570071 8505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:56.570823 8505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:56.570878 8505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.570916 8505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:56.570937 8505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.579327 8505 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:40449
I20260808 10:39:56.579396 8645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:40449 every 8 connection(s)
I20260808 10:39:56.580236 8505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:56.587739 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8505
I20260808 10:39:56.587865 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:39:56.588191 1801 external_mini_cluster.cc:1428] Setting key b54b29c0994490f5b73ea876c902a80a
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:39:56.594758 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:56.582033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:52057 (local address 127.1.194.126:42389)
0808 10:39:56.582198 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:56.582202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:56.583021 (+ 819us) server_negotiation.cc:407] Connection header received
0808 10:39:56.584437 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:56.584440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:56.584499 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:56.584630 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:56.585845 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.586388 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.587533 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.587780 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.587929 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:56.590786 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:56.590793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:56.590798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:56.590830 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:56.592463 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:56.593161 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:56.593165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:56.593167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:56.593253 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:56.593668 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:56.593672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:56.593674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:56.593862 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:56.593973 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:56.594462 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 10:39:56.594589 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:39:56.595556 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:56.582396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:52057)
0808 10:39:56.582866 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:56.582908 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:56.584218 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:56.584798 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:56.584804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:56.585157 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:56.585701 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:56.585717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.586517 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.586523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:56.587377 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:56.587386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.588004 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.588009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:56.588230 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:56.588955 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:56.588979 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:56.590622 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:56.592627 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:56.592634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:56.592649 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:56.593045 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:56.593352 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:56.593355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:56.593357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:56.593571 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:56.593970 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:56.593977 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:56.594222 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 10:39:56.594405 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 10:39:56.594738 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":109,"threads_started":1}
I20260808 10:39:56.596936 8646 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42389
I20260808 10:39:56.597184 8646 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:56.597749 8646 heartbeater.cc:511] Master 127.1.194.126:42389 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:39:56.599291 8440 ts_manager.cc:194] Registered new tserver with Master: 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449)
I20260808 10:39:56.600461 8440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:52057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab.
Entry for principal oryx/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab.
I20260808 10:39:56.621912 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42389
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:56.742710 8651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:56.743042 8651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:56.743139 8651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:56.747635 8651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:56.747725 8651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:56.747813 8651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:56.753221 8651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:56.754349 8651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:56.755402 8651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:56.765422 8659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:56.765646 8657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:56.765625 8656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:56.765637 8651 server_base.cc:1034] running on GCE node
I20260808 10:39:56.766224 8651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:56.766880 8651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:56.768049 8651 hybrid_clock.cc:648] HybridClock initialized: now 1786185596768031 us; error 25 us; skew 500 ppm
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:56.771173 8651 init.cc:377] Logged in from keytab as oryx/127.1.194.66@KRBTEST.COM (short username oryx)
I20260808 10:39:56.772059 8651 webserver.cc:460] Webserver started at http://127.1.194.66:42847/ using document root <none> and password file <none>
I20260808 10:39:56.772615 8651 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:56.772676 8651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:56.772886 8651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:56.774978 8651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "82f336cdb47444bc92539988dac37388"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "aa2daa51de5c3e1c2f1eb6b59623099d"
server_key_iv: "dcc86fc27bbba806dc003b18cbb0a23c"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.775538 8651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "82f336cdb47444bc92539988dac37388"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "aa2daa51de5c3e1c2f1eb6b59623099d"
server_key_iv: "dcc86fc27bbba806dc003b18cbb0a23c"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.779371 8651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 10:39:56.781975 8667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.783228 8651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:56.783360 8651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "82f336cdb47444bc92539988dac37388"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "aa2daa51de5c3e1c2f1eb6b59623099d"
server_key_iv: "dcc86fc27bbba806dc003b18cbb0a23c"
server_key_version: "encryptionkey@0"
I20260808 10:39:56.783470 8651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:56.811616 8651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:56.814823 8651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:56.815032 8651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:56.815640 8651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:56.816489 8651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:56.816569 8651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.816658 8651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:56.816700 8651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:56.826480 8651 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:39121
I20260808 10:39:56.826614 8791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:39121 every 8 connection(s)
I20260808 10:39:56.827422 8651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:56.830596 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8651
I20260808 10:39:56.830713 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:39:56.831076 1801 external_mini_cluster.cc:1428] Setting key 8007807bf476143605349c9fbc0923b7
Aug 08 10:39:56 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:39:56.842532 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:56.829352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:40009 (local address 127.1.194.126:42389)
0808 10:39:56.829584 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:56.829594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:56.830459 (+ 865us) server_negotiation.cc:407] Connection header received
0808 10:39:56.832067 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:56.832070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:56.832126 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:56.832204 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:56.833450 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.834024 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.835166 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.835424 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.835551 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:56.838400 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:56.838406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:56.838413 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:56.838450 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:56.840200 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:56.840719 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:56.840724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:56.840728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:56.840829 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:56.841184 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:56.841190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:56.841192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:56.841459 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:56.841599 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:56.842200 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 10:39:56.842343 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:39:56.843475 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:56.829862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:40009)
0808 10:39:56.830293 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:56.830354 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:56.831853 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:56.832397 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:56.832406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:56.832810 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:56.833301 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:56.833315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.834161 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.834169 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:56.835037 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:56.835046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:56.835581 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:56.835586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:56.835761 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:56.836358 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:56.836383 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:56.838212 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:56.840326 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:56.840330 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:56.840342 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:56.840608 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:56.840934 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:56.840936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:56.840937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:56.841084 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:56.841628 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:56.841634 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:56.841904 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 10:39:56.842098 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:39:56.842424 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
I20260808 10:39:56.844700 8792 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42389
I20260808 10:39:56.844960 8792 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:56.845461 8792 heartbeater.cc:511] Master 127.1.194.126:42389 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.67@KRBTEST.COM" created.
I20260808 10:39:56.846719 8440 ts_manager.cc:194] Registered new tserver with Master: 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:39:56.847617 8440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:40009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab.
Entry for principal oryx/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab.
I20260808 10:39:56.865890 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42389
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:56.980792 8797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:56.981082 8797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:56.981163 8797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:56.985308 8797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:56.985392 8797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:56.985527 8797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:56.991025 8797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:56.992116 8797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:56.993151 8797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:57.002815 8804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.002808 8803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.002818 8806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:57.003381 8797 server_base.cc:1034] running on GCE node
I20260808 10:39:57.003796 8797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:57.004427 8797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:57.005594 8797 hybrid_clock.cc:648] HybridClock initialized: now 1786185597005563 us; error 49 us; skew 500 ppm
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:57.008400 8797 init.cc:377] Logged in from keytab as oryx/127.1.194.67@KRBTEST.COM (short username oryx)
I20260808 10:39:57.009250 8797 webserver.cc:460] Webserver started at http://127.1.194.67:44921/ using document root <none> and password file <none>
I20260808 10:39:57.009886 8797 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.009984 8797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.010231 8797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:57.012390 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "2cb41e7dd76845ba8af80e6db0b66d68"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "6b6a4e6b60eb5c306a36556f84c3f114"
server_key_iv: "40f8ef88dd4e4c59daa3d770d670fe6e"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.012969 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "2cb41e7dd76845ba8af80e6db0b66d68"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "6b6a4e6b60eb5c306a36556f84c3f114"
server_key_iv: "40f8ef88dd4e4c59daa3d770d670fe6e"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.016762 8797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:39:57.019284 8813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.020511 8797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:57.020650 8797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "2cb41e7dd76845ba8af80e6db0b66d68"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "6b6a4e6b60eb5c306a36556f84c3f114"
server_key_iv: "40f8ef88dd4e4c59daa3d770d670fe6e"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.020766 8797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.043623 8797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:57.046864 8797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:57.047079 8797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:57.047695 8797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:57.048555 8797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:57.048633 8797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.048708 8797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:57.048760 8797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.058266 8797 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:39981
I20260808 10:39:57.058295 8935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:39981 every 8 connection(s)
I20260808 10:39:57.059154 8797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:57.063772 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8797
I20260808 10:39:57.063905 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:39:57.064235 1801 external_mini_cluster.cc:1428] Setting key 414064414ac1761a401c7f45aee9db3e
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:39:57.074095 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.061120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:43085 (local address 127.1.194.126:42389)
0808 10:39:57.061266 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.061270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.062188 (+ 918us) server_negotiation.cc:407] Connection header received
0808 10:39:57.063965 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.063968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.064058 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.064149 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.065476 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.066302 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.067571 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.067819 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.067937 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.070586 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.070590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.070592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.070620 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.072102 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.072597 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.072600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.072601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.072688 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.073021 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.073024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.073025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.073211 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.073295 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.073868 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.073970 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:39:57.074880 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.061444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.67:43085)
0808 10:39:57.061990 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.062053 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.063337 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.064365 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.064376 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.064817 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.065298 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.065311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.066447 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.066450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.067439 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.067450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.067963 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.067966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.068119 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.068692 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.068708 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.070365 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.072218 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.072223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.072235 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.072486 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.072781 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.072783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.072784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.072912 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.073313 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.073319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.073632 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.073796 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.074047 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":131,"threads_started":1}
I20260808 10:39:57.076063 8936 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42389
I20260808 10:39:57.076313 8936 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:57.076813 8936 heartbeater.cc:511] Master 127.1.194.126:42389 requested a full tablet report, sending...
I20260808 10:39:57.077945 8440 ts_manager.cc:194] Registered new tserver with Master: 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981)
I20260808 10:39:57.078555 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:39:57.078627 8440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:43085
WARNING: no policy specified for oryx/127.1.194.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.125.keytab.
Entry for principal oryx/127.1.194.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.125.keytab.
I20260808 10:39:57.109268 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:40471
--webserver_interface=127.1.194.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.125.keytab
--principal=oryx/127.1.194.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42389,127.1.194.125:40471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:57.220005 8942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:57.220301 8942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:57.220343 8942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:57.224509 8942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:57.224582 8942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:57.224602 8942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:57.224619 8942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:57.224635 8942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:57.229981 8942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42389,127.1.194.125:40471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.125.keytab
--principal=oryx/127.1.194.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:40471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.194.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
W20260808 10:39:57.230180 8942 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.194.126:42389,127.1.194.125:40471'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 10:39:57.230999 8942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:57.231920 8942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:57.240149 8950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.240144 8948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.240192 8947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:57.240412 8942 server_base.cc:1034] running on GCE node
I20260808 10:39:57.241082 8942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:57.242362 8942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:57.243564 8942 hybrid_clock.cc:648] HybridClock initialized: now 1786185597243548 us; error 33 us; skew 500 ppm
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:57.246425 8942 init.cc:377] Logged in from keytab as oryx/127.1.194.125@KRBTEST.COM (short username oryx)
I20260808 10:39:57.247373 8942 webserver.cc:460] Webserver started at http://127.1.194.125:32775/ using document root <none> and password file <none>
I20260808 10:39:57.247920 8942 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.248013 8942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.248251 8942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:57.250401 8942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/instance:
uuid: "cd7313f308b646f1af7676abc36a516e"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "af6edcfacafcde15e8091cbc34c1a9f9"
server_key_iv: "1d548a212a7162c9b41aae3c7413e2ce"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.250957 8942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal/instance:
uuid: "cd7313f308b646f1af7676abc36a516e"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "af6edcfacafcde15e8091cbc34c1a9f9"
server_key_iv: "1d548a212a7162c9b41aae3c7413e2ce"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.254730 8942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 10:39:57.257205 8959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.258886 8942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:39:57.259073 8942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
uuid: "cd7313f308b646f1af7676abc36a516e"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "af6edcfacafcde15e8091cbc34c1a9f9"
server_key_iv: "1d548a212a7162c9b41aae3c7413e2ce"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.259214 8942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.277688 8942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:57.280917 8942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:57.281078 8942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:57.288571 8942 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.125:40471
I20260808 10:39:57.288878 9015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.125:40471 every 8 connection(s)
I20260808 10:39:57.289628 8942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
I20260808 10:39:57.292626 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:57.296314 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 8942
I20260808 10:39:57.296617 9016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.125@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
I20260808 10:39:57.310850 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.298758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.1.194.126:42389)
0808 10:39:57.298873 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.298877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.299265 (+ 388us) server_negotiation.cc:407] Connection header received
0808 10:39:57.299719 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.299722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.299778 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.299869 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.301025 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.301827 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.303317 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.303629 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.303741 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.306333 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.306337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.306340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.306370 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.308278 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.309049 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.309053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.309055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.309151 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.309622 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.309625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.309628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.309896 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.310003 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.310533 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.310668 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:39:57.311931 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.298024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.1.194.125:40471)
0808 10:39:57.298430 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.298453 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.298486 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:39:57.299377 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.299398 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.299633 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.299825 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.300662 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.302004 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.302842 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.303384 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.303532 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.306866 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.306871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.306881 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.306901 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.308904 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.309363 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.309368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.309370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.309464 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.309795 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.309799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.309801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.310241 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.310454 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.310681 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.311123 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":260,"thread_start_us":134,"threads_started":1}
I20260808 10:39:57.311987 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.298667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:43586)
0808 10:39:57.299130 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.299180 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.299504 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.300040 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.300046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.300516 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.300865 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.300879 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.301963 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.301966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.303187 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.303197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.303799 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.303803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.303942 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.304482 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.304497 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.306130 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.308403 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.308412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.308423 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.308910 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.309269 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.309273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.309275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.309468 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.310092 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.310099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.310261 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.311005 (+ 744us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.311202 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":3,"thread_start_us":139,"threads_started":1}
I20260808 10:39:57.313617 9016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.125@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
I20260808 10:39:57.322024 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.314174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40126)
0808 10:39:57.314283 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.314298 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.314379 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.314590 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.314593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.314738 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.314835 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.314840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.315574 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.315576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.316105 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.316113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.317113 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.317116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.317196 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.317790 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.317802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.319018 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.320523 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.320529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.320532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.320724 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.320989 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.320994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.320995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.321076 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.321444 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.321448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.321588 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.321694 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.321841 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:39:57.322024 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.314301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.1.194.125:40471)
0808 10:39:57.314395 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.314399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.314412 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:39:57.314451 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.314453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.314484 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.314544 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.314941 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.315453 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.316201 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.316974 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.317065 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.319144 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.319148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.319150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.319171 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.320414 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.320817 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.320823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.320825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.320898 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.321171 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.321174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.321175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.321344 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.321444 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.321750 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.321893 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":36}
I20260808 10:39:57.325228 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Bootstrap starting.
I20260808 10:39:57.327651 9016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:57.328444 9016 log.cc:824] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:57.330528 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: No bootstrap required, opened a new log
I20260808 10:39:57.333472 9016 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } }
I20260808 10:39:57.333734 9016 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:57.333814 9016 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7313f308b646f1af7676abc36a516e, State: Initialized, Role: FOLLOWER
I20260808 10:39:57.334275 9016 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } }
I20260808 10:39:57.334775 9022 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } } }
I20260808 10:39:57.334913 9022 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:57.335453 9016 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:39:57.339845 9034 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:57.339936 9034 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:57.342793 8942 master_runner.cc:428] Detected that this master cd7313f308b646f1af7676abc36a516e is joining an existing cluster
I20260808 10:39:57.342865 8942 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.194.125:40471
I20260808 10:39:57.353181 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.345264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43588 (local address 127.1.194.126:42389)
0808 10:39:57.345390 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.345395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.345430 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:39:57.345616 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.345619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.345689 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.345762 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.346227 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.346976 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.348157 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.348470 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.348602 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.349784 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.349787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.349790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.349815 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.351439 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.351917 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.351920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.351922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.352042 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.352317 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.352321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.352323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.352535 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.352639 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.352895 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.353007 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:39:57.353358 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.344897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:43588)
0808 10:39:57.345360 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.345373 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.345441 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.345785 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.345787 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.346001 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.346113 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.346119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.347116 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.347119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.348026 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.348037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.348602 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.348606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.348706 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.349197 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.349213 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.349654 (+ 441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.351559 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.351564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.351566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.351811 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.352131 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.352133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.352135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.352223 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.352640 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.352644 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.352738 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.352843 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.352990 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":116,"threads_started":1}
I20260808 10:39:57.353243 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.344950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.1.194.125:40471)
0808 10:39:57.345078 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.345083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.345684 (+ 601us) server_negotiation.cc:407] Connection header received
0808 10:39:57.345842 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.345845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.345880 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.345924 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.346530 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.347198 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.347981 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.348257 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.348350 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.349733 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.349737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.349739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.349760 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.351577 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.352036 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.352039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.352040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.352122 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.352395 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.352398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.352400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.352595 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.352685 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.352944 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.353062 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:39:57.354672 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.345090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40128)
0808 10:39:57.345595 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.345608 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.345707 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.346053 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.346056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.346291 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.346403 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.346410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.347307 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.347310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.347867 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.347874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.348382 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.348385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.348477 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.349065 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.349082 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.349601 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.351700 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.351702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.351704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.351941 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.352196 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.352201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.352203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.352317 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.352719 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.352723 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.352797 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.354376 (+ 1579us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.354505 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"spinlock_wait_cycles":32640,"thread_start_us":104,"threads_started":1}
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.126@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
I20260808 10:39:57.368724 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.358386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.1.194.125:40471)
0808 10:39:57.358493 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.358497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.358840 (+ 343us) server_negotiation.cc:407] Connection header received
0808 10:39:57.359024 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.359026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.359084 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.359167 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.359967 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.360837 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.361798 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.362175 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.362274 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.364720 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.364723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.364726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.364751 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.366681 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.367252 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.367259 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.367263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.367360 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.367667 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.367671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.367674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.367928 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.368052 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.368428 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.368560 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:39:57.368739 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.358336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40136)
0808 10:39:57.358707 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.358756 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.358890 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.359296 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.359305 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.359732 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.359855 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.359862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.360954 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.360957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.361690 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.361700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.362301 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.362304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.362410 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.362983 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.362996 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.364564 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.366805 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.366815 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.366822 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.367145 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.367462 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.367466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.367468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.367563 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.368066 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.368076 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.368202 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.368373 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.368581 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":132,"threads_started":1}
I20260808 10:39:57.370626 8440 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.194.125:40471: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cd7313f308b646f1af7676abc36a516e"
member_type: NON_VOTER
last_known_addr {
host: "127.1.194.125"
port: 40471
}
attrs {
promote: true
}
}
dest_uuid: "643614d186614f9ea5572a49969ba2e9"
cas_config_opid_index: -1
I20260808 10:39:57.371491 8440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } }
I20260808 10:39:57.373708 9047 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cd7313f308b646f1af7676abc36a516e (127.1.194.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } }
I20260808 10:39:57.374392 8988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:57.375584 9047 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.194.125:40471
I20260808 10:39:57.375588 8988 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Refusing update from remote peer 643614d186614f9ea5572a49969ba2e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
WARNING: no policy specified for oryx/127.1.194.124@KRBTEST.COM; defaulting to no policy
I20260808 10:39:57.376125 8942 master.cc:537] Master@127.1.194.125:40471 shutting down...
I20260808 10:39:57.376564 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } }
I20260808 10:39:57.376674 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:57.376861 9047 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:39:57.379024 8425 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 10:39:57.380084 8425 consensus_peers.cc:487] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 -> Peer cd7313f308b646f1af7676abc36a516e (127.1.194.125:40471): Couldn't send request to peer cd7313f308b646f1af7676abc36a516e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:39:57.381280 8942 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:57.381615 8942 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:57.381738 8942 tablet_replica.cc:330] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: stopping tablet replica
I20260808 10:39:57.387002 8942 master.cc:552] Master@127.1.194.125:40471 shutdown complete.
I20260808 10:39:57.387070 8942 master_runner.cc:305] Clearing existing system tablet
I20260808 10:39:57.387293 8942 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.387367 8942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.124@KRBTEST.COM" created.
I20260808 10:39:57.388883 8942 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:39:57.389909 9054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.390199 8942 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.390439 8942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:39:57.390516 8942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
uuid: "cd7313f308b646f1af7676abc36a516e"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "af6edcfacafcde15e8091cbc34c1a9f9"
server_key_iv: "1d548a212a7162c9b41aae3c7413e2ce"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.393090 8942 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:57.396353 8942 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:39:57.396507 8942 log.cc:1196] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 10:39:57.396850 8942 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Deleting consensus metadata
I20260808 10:39:57.397073 8942 master_runner.cc:315] Copying system tablet from 127.1.194.126:42389
I20260808 10:39:57.398626 8942 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: tablet copy: Beginning tablet copy session from remote peer at address 127.1.194.126:42389
I20260808 10:39:57.408622 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.399286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:43596)
0808 10:39:57.399723 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.399738 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.399910 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.400194 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.400197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.400396 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.400509 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.400516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.401348 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.401352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.402431 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.402443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.402915 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.402919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.403018 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.404105 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.404120 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.404603 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.406825 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.406828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.406832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.407049 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.407418 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.407425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.407435 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.407593 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.408069 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.408071 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.408159 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.408247 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.408363 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":99,"threads_started":1}
I20260808 10:39:57.408655 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.399386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.1.194.126:42389)
0808 10:39:57.399535 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.399543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.399830 (+ 287us) server_negotiation.cc:407] Connection header received
0808 10:39:57.400018 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.400021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.400079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.400177 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.400636 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.401220 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.402561 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.402785 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.402875 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.404748 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.404751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.404753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.404778 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.406643 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.407154 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.407158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.407160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.407251 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.407692 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.407695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.407697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.407945 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.408078 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.408359 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.408496 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.124.keytab.
Entry for principal oryx/127.1.194.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.124.keytab.
I20260808 10:39:57.409628 8462 tablet_copy_service.cc:140] P 643614d186614f9ea5572a49969ba2e9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cd7313f308b646f1af7676abc36a516e ({username='oryx', principal='oryx/127.1.194.125@KRBTEST.COM'} at 127.0.0.1:43596)
I20260808 10:39:57.409822 8462 tablet_copy_service.cc:161] P 643614d186614f9ea5572a49969ba2e9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cd7313f308b646f1af7676abc36a516e at {username='oryx', principal='oryx/127.1.194.125@KRBTEST.COM'} at 127.0.0.1:43596: session id = cd7313f308b646f1af7676abc36a516e-00000000000000000000000000000000
I20260808 10:39:57.410007 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:46267
--webserver_interface=127.1.194.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.124.keytab
--principal=oryx/127.1.194.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:39:57.411406 8462 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:39:57.413447 8942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:39:57.416705 8942 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: tablet copy: Starting download of 0 data blocks...
I20260808 10:39:57.416941 8942 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: tablet copy: Starting download of 1 WAL segments...
I20260808 10:39:57.419111 8942 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:39:57.421523 8462 tablet_copy_service.cc:342] P 643614d186614f9ea5572a49969ba2e9: Request end of tablet copy session cd7313f308b646f1af7676abc36a516e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.194.125@KRBTEST.COM'} at 127.0.0.1:43596
I20260808 10:39:57.421685 8462 tablet_copy_service.cc:433] P 643614d186614f9ea5572a49969ba2e9: ending tablet copy session cd7313f308b646f1af7676abc36a516e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cd7313f308b646f1af7676abc36a516e
I20260808 10:39:57.430115 8942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:57.431902 9064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.431913 9063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.432673 9066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:57.433002 8942 server_base.cc:1034] running on GCE node
I20260808 10:39:57.433172 8942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:57.433457 8942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:57.434634 8942 hybrid_clock.cc:648] HybridClock initialized: now 1786185597434612 us; error 38 us; skew 500 ppm
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:57.436252 8942 init.cc:377] Logged in from keytab as oryx/127.1.194.125@KRBTEST.COM (short username oryx)
I20260808 10:39:57.436743 8942 webserver.cc:460] Webserver started at http://127.1.194.125:37283/ using document root <none> and password file <none>
I20260808 10:39:57.436972 8942 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.437019 8942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.438488 8942 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:57.439568 9073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.439845 8942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:57.439949 8942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
uuid: "cd7313f308b646f1af7676abc36a516e"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "af6edcfacafcde15e8091cbc34c1a9f9"
server_key_iv: "1d548a212a7162c9b41aae3c7413e2ce"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.440068 8942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.454244 8942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:57.456770 8942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:57.461851 8942 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.125:40471
I20260808 10:39:57.462103 9129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.125:40471 every 8 connection(s)
I20260808 10:39:57.463267 8942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
I20260808 10:39:57.465587 9130 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:39:57.466769 9130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Bootstrap starting.
I20260808 10:39:57.478415 9130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:39:57.478783 9130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Bootstrap complete.
I20260808 10:39:57.479415 9130 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } }
I20260808 10:39:57.479562 9130 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cd7313f308b646f1af7676abc36a516e, State: Initialized, Role: LEARNER
I20260808 10:39:57.479671 9130 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } }
I20260808 10:39:57.480048 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } }
I20260808 10:39:57.480182 9130 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:57.480276 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: LEARNER
I20260808 10:39:57.483441 9148 catalog_manager.cc:1259] Loaded cluster ID: ed16ecec505c4e54809e973b9d4b105b
I20260808 10:39:57.483584 9148 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: loading cluster ID for follower catalog manager: success
I20260808 10:39:57.485852 9148 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: acquiring CA information for follower catalog manager: success
I20260808 10:39:57.487277 9148 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 10:39:57.556785 9062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:57.557129 9062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:57.557248 9062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:57.562259 9062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:57.562368 9062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:57.562407 9062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:57.562450 9062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:57.562485 9062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:57.568549 9062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.124.keytab
--principal=oryx/127.1.194.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:46267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.194.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:57.569690 9062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:57.570662 9062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:57.580605 9153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.580678 9154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.580643 9156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:57.581269 9062 server_base.cc:1034] running on GCE node
I20260808 10:39:57.581995 9062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:57.583410 9062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:57.584614 9062 hybrid_clock.cc:648] HybridClock initialized: now 1786185597584580 us; error 54 us; skew 500 ppm
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:57.587698 9062 init.cc:377] Logged in from keytab as oryx/127.1.194.124@KRBTEST.COM (short username oryx)
I20260808 10:39:57.588675 9062 webserver.cc:460] Webserver started at http://127.1.194.124:36809/ using document root <none> and password file <none>
I20260808 10:39:57.589208 9062 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.589268 9062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.589425 9062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:57.591445 9062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/instance:
uuid: "96a464f56b6248c99f4cc594c018d2da"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "f929c4d906cdc122b769fd4af2cee498"
server_key_iv: "3a574abf26b1ea28ea0d753d60eb82a4"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.592029 9062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal/instance:
uuid: "96a464f56b6248c99f4cc594c018d2da"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "f929c4d906cdc122b769fd4af2cee498"
server_key_iv: "3a574abf26b1ea28ea0d753d60eb82a4"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.595968 9062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 10:39:57.598385 9163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.599671 9062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:57.599787 9062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
uuid: "96a464f56b6248c99f4cc594c018d2da"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "f929c4d906cdc122b769fd4af2cee498"
server_key_iv: "3a574abf26b1ea28ea0d753d60eb82a4"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.599926 9062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.603248 8646 heartbeater.cc:503] Master 127.1.194.126:42389 was elected leader, sending a full tablet report...
I20260808 10:39:57.616925 9062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:57.620501 9062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:57.620723 9062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:57.631834 9062 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.124:46267
I20260808 10:39:57.632227 9218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.124:46267 every 8 connection(s)
I20260808 10:39:57.633076 9062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
I20260808 10:39:57.636113 9219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:57.639038 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 9062
I20260808 10:39:57.641243 9219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.124@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.194.124@KRBTEST.COM
I20260808 10:39:57.654577 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.643928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.1.194.126:42389)
0808 10:39:57.644096 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.644101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.644420 (+ 319us) server_negotiation.cc:407] Connection header received
0808 10:39:57.644546 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.644549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.644610 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.644679 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.645591 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.646097 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.647144 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.647514 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.647630 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.650392 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.650395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.650397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.650424 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.652240 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.652816 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.652819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.652820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.652898 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.653351 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.653359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.653360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.653656 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.653766 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.654283 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.654403 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:39:57.655615 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.640547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.1.194.124:46267)
0808 10:39:57.641062 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.641094 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.641131 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:39:57.642116 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.642142 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.642591 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.642951 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.643970 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.645394 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.646572 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.647119 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.647279 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.650903 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.650910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.650924 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.650948 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.653123 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.653635 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.653641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.653642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.653728 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.654019 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.654022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.654024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.654253 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.654507 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.655026 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.655148 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":105,"threads_started":1}
I20260808 10:39:57.655644 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.643805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:43602)
0808 10:39:57.644282 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.644329 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.644450 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.644750 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.644755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.645194 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.645377 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.645390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.646232 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.646236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.647014 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.647023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.647665 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.647668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.647795 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.648292 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.648315 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.650225 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.652370 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.652378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.652389 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.652700 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.653018 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.653024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.653026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.653240 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.653765 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.653772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.654039 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.654241 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.654556 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":210,"threads_started":1}
I20260808 10:39:57.657786 9219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.124@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
I20260808 10:39:57.667073 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.658307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40148)
0808 10:39:57.658434 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.658449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.658592 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.659146 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.659150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.659325 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.659442 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.659447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.660584 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.660587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.661191 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.661200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.661872 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.661874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.661954 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.662490 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.662501 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.663829 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.665674 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.665676 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.665678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.665866 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.666144 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.666147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.666148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.666255 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.666662 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.666665 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.666747 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.666833 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.666939 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 10:39:57.667375 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.658436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.1.194.125:40471)
0808 10:39:57.658830 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.658834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.658850 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:39:57.658895 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.658897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.658951 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.659051 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.659591 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.660359 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.661328 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.661734 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.661849 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.663988 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.663991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.663993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.664019 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.665463 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.665948 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.665951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.665952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.666030 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.666355 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.666357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.666359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.666535 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.666644 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.666949 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.667090 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":176,"threads_started":1}
I20260808 10:39:57.668370 9219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.124@KRBTEST.COM for oryx/127.1.194.124@KRBTEST.COM
I20260808 10:39:57.676497 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.668910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50122)
0808 10:39:57.669035 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.669053 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.669122 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.669311 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.669313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.669444 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.669609 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.669615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.670708 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.670710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.671201 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.671208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.671705 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.671708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.671766 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.672222 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.672232 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.673439 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.675248 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.675250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.675252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.675431 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.675671 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.675673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.675674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.675744 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.676059 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.676061 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.676133 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.676227 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.676327 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:39:57.676506 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.668986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.1.194.124:46267)
0808 10:39:57.669129 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.669132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.669165 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:39:57.669193 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.669195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.669227 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.669282 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.669732 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.670601 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.671297 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.671602 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.671688 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.673621 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.673624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.673626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.673646 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.675147 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.675513 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.675515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.675516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.675588 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.675819 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.675821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.675822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.675980 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.676082 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.676262 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.676379 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":83}
I20260808 10:39:57.679668 9219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Bootstrap starting.
I20260808 10:39:57.682132 9219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:57.683079 9219 log.cc:824] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:57.685141 9219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: No bootstrap required, opened a new log
I20260808 10:39:57.688504 9219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } }
I20260808 10:39:57.688741 9219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:57.688840 9219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a464f56b6248c99f4cc594c018d2da, State: Initialized, Role: FOLLOWER
I20260808 10:39:57.689325 9219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } }
I20260808 10:39:57.689853 9226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } } }
I20260808 10:39:57.690006 9226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:57.690594 9219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:39:57.696185 9239 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:57.696305 9239 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:57.700538 9062 master_runner.cc:428] Detected that this master 96a464f56b6248c99f4cc594c018d2da is joining an existing cluster
I20260808 10:39:57.700628 9062 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.194.124:46267
I20260808 10:39:57.709141 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8505
I20260808 10:39:57.722551 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:40449
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:42389
--webserver_port=40051
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:39:57.738111 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.704639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.1.194.125:40471)
0808 10:39:57.704791 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.704797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.704812 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:39:57.704855 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.704858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.704910 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.704979 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.705775 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.706733 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.708236 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.708634 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.708724 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.711177 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.711182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.711185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.711525 (+ 340us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.713777 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.734375 (+ 20598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.734380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.734382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.734499 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.734820 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.734824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.734826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.735075 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.735159 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.737760 (+ 2601us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.737924 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:39:57.738804 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.704656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.1.194.124:46267)
0808 10:39:57.704777 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.704781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.705770 (+ 989us) server_negotiation.cc:407] Connection header received
0808 10:39:57.705903 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.705935 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.705979 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.706041 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.706699 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.707421 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.710072 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.710522 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.710637 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.735008 (+ 24371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.735012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.735014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.735037 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.737006 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.737613 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.737616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.737618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.737696 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.737932 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.737935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.737937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.738121 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.738201 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.738473 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.738587 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:39:57.738677 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.703647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40150)
0808 10:39:57.704252 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.704271 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.704374 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.705117 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.705121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.705399 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.705607 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.705621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.706894 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.706897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.708078 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.708089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.709013 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.709017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.709182 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.710408 (+ 1226us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.710425 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.711002 (+ 577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.713931 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.713934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.713948 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.734176 (+ 20228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.734628 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.734631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.734633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.734722 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.735216 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.735219 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.735642 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.738331 (+ 2689us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.738496 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":28,"thread_start_us":118,"threads_started":1}
I20260808 10:39:57.739006 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.704207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50132)
0808 10:39:57.705701 (+ 1494us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.705715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.705819 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.706194 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.706197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.706460 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.706587 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.706595 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.707550 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.707554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.709954 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.709968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.710818 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.710825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.711023 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.716786 (+ 5763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.716808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.734382 (+ 17574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.737119 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.737122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.737124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.737506 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.737784 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.737787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.737789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.737858 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.738251 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.738255 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.738337 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.738694 (+ 357us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.738815 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1417,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
I20260808 10:39:57.742100 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.703706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:43608)
0808 10:39:57.705247 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.705259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.705419 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.705843 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.705849 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.706260 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.706462 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.706470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.707562 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.707567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.716527 (+ 8960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.716545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.717379 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.717383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.717512 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.718030 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.718044 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.734834 (+ 16790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.739088 (+ 4254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.739091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.739094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.740484 (+ 1390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.740828 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.740831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.740833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.741042 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.741576 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.741579 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.741668 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.741762 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.741907 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1487,"mutex_wait_us":7368,"thread_start_us":157,"threads_started":1}
I20260808 10:39:57.742208 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.703752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.1.194.126:42389)
0808 10:39:57.703892 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.703897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.705381 (+ 1484us) server_negotiation.cc:407] Connection header received
0808 10:39:57.705578 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.705583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.705638 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.705717 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.706615 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.707438 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.716738 (+ 9300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.717131 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.717246 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.735425 (+ 18179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.735430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.735432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.735467 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.737273 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.740614 (+ 3341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.740619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.740621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.740716 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.741149 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.741153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.741155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.741405 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.741519 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.741888 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.742026 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185596, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.126@KRBTEST.COM for oryx/127.1.194.124@KRBTEST.COM
I20260808 10:39:57.755717 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.746334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50142)
0808 10:39:57.746451 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.746467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.746548 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.746887 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.746889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.747093 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.747206 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.747213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.748276 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.748279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.749168 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.749178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.749798 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.749802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.749903 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.750423 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.750439 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.751977 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.753956 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.753960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.753962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.754213 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.754580 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.754583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.754585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.754678 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.755168 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.755171 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.755275 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.755385 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.755537 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:39:57.756155 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.746505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.1.194.124:46267)
0808 10:39:57.746627 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.746631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.746644 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:39:57.746716 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.746720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.746773 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.746854 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.747348 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.748159 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.749297 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.749676 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.749782 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.752125 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.752129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.752131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.752158 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.753834 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.754320 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.754327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.754330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.754419 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.754804 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.754807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.754809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.755068 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.755167 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.755460 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.755654 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:57.757297 8440 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.194.124:46267: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "96a464f56b6248c99f4cc594c018d2da"
member_type: NON_VOTER
last_known_addr {
host: "127.1.194.124"
port: 46267
}
attrs {
promote: true
}
}
dest_uuid: "643614d186614f9ea5572a49969ba2e9"
cas_config_opid_index: 5
I20260808 10:39:57.758159 8440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } }
I20260808 10:39:57.759219 9047 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 96a464f56b6248c99f4cc594c018d2da (127.1.194.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.760687 9047 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.194.124:46267
I20260808 10:39:57.760748 9192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:57.760874 9047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.760962 9047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:57.761636 9062 master.cc:537] Master@127.1.194.124:46267 shutting down...
I20260808 10:39:57.762588 9192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Refusing update from remote peer 643614d186614f9ea5572a49969ba2e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:39:57.763787 9048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:39:57.766007 8424 consensus_peers.cc:487] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 -> Peer 96a464f56b6248c99f4cc594c018d2da (127.1.194.124:46267): Couldn't send request to peer 96a464f56b6248c99f4cc594c018d2da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:39:57.767568 9062 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:57.767897 9062 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:57.768013 9062 tablet_replica.cc:330] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: stopping tablet replica
I20260808 10:39:57.770548 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.759638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.1.194.125:40471)
0808 10:39:57.759757 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.759762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.763721 (+ 3959us) server_negotiation.cc:407] Connection header received
0808 10:39:57.763924 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.763928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.764010 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.764077 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:57.766376 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.767166 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.769048 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.769728 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.769810 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.769972 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.770331 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.770417 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:39:57.771093 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.759396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40156)
0808 10:39:57.763638 (+ 4242us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.763654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.763802 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.764336 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.764339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.764527 (+ 188us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:57.764623 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.764630 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.767316 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.767319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.768888 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.768900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.770072 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.770075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.770170 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.770216 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.770874 (+ 658us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.770972 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4160}
I20260808 10:39:57.771809 9103 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 LEARNER]: Refusing update from remote peer 643614d186614f9ea5572a49969ba2e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:39:57.772296 9048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 10:39:57.773393 9062 master.cc:552] Master@127.1.194.124:46267 shutdown complete.
I20260808 10:39:57.773434 9062 master_runner.cc:305] Clearing existing system tablet
I20260808 10:39:57.773723 9062 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.773788 9062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.774257 9103 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 96a464f56b6248c99f4cc594c018d2da (127.1.194.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.776062 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.776168 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: LEARNER
I20260808 10:39:57.776099 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 643614d186614f9ea5572a49969ba2e9. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: NON_VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: true } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.776229 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: LEARNER
I20260808 10:39:57.778527 9062 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 10:39:57.778823 9048 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: attempting to promote NON_VOTER cd7313f308b646f1af7676abc36a516e to VOTER
I20260808 10:39:57.779286 9048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } }
I20260808 10:39:57.780613 9102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 LEARNER]: Refusing update from remote peer 643614d186614f9ea5572a49969ba2e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 10:39:57.781092 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.780275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50152)
0808 10:39:57.780925 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.781010 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:46267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564}
I20260808 10:39:57.781050 9047 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 10:39:57.782063 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.781613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50162)
0808 10:39:57.781937 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.781990 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:46267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":106,"threads_started":1}
W20260808 10:39:57.782147 8424 consensus_peers.cc:487] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 -> Peer 96a464f56b6248c99f4cc594c018d2da (127.1.194.124:46267): Couldn't send request to peer 96a464f56b6248c99f4cc594c018d2da. Status: Network error: Client connection negotiation failed: client connection to 127.1.194.124:46267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:39:57.783034 9045 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cd7313f308b646f1af7676abc36a516e (127.1.194.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.783516 9102 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cd7313f308b646f1af7676abc36a516e (127.1.194.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.784622 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.784734 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:57.784852 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.784914 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:57.785650 9252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.786875 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.787001 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:57.789712 9062 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.789870 9062 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260808 10:39:57.789915 9062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
uuid: "96a464f56b6248c99f4cc594c018d2da"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "f929c4d906cdc122b769fd4af2cee498"
server_key_iv: "3a574abf26b1ea28ea0d753d60eb82a4"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.792104 9062 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:57.794783 9062 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:39:57.794884 9062 log.cc:1196] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 10:39:57.795156 9062 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Deleting consensus metadata
I20260808 10:39:57.795346 9062 master_runner.cc:315] Copying system tablet from 127.1.194.126:42389
I20260808 10:39:57.796653 9062 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: tablet copy: Beginning tablet copy session from remote peer at address 127.1.194.126:42389
I20260808 10:39:57.806736 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.797211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:43620)
0808 10:39:57.797537 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.797555 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.797834 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.798169 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.798174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.798403 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.798548 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.798555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.799650 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.799654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.801151 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.801165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.801867 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.801874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.801972 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.802549 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.802564 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.803011 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.805034 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.805037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.805040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.805301 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.805640 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.805642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.805645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.805746 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.806176 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.806179 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.806284 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.806395 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.806544 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
I20260808 10:39:57.806746 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.797613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.1.194.126:42389)
0808 10:39:57.797736 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.797740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.797758 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:39:57.797971 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.797975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.798029 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.798115 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.798691 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.799523 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.801328 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.801751 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.801862 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.803167 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.803171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.803174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.803202 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.804907 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.805406 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.805410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.805412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.805540 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.805845 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.805848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.805850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.806078 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.806175 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.806465 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.806586 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:39:57.807498 8462 tablet_copy_service.cc:140] P 643614d186614f9ea5572a49969ba2e9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 96a464f56b6248c99f4cc594c018d2da ({username='oryx', principal='oryx/127.1.194.124@KRBTEST.COM'} at 127.0.0.1:43620)
I20260808 10:39:57.807595 8462 tablet_copy_service.cc:161] P 643614d186614f9ea5572a49969ba2e9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 96a464f56b6248c99f4cc594c018d2da at {username='oryx', principal='oryx/127.1.194.124@KRBTEST.COM'} at 127.0.0.1:43620: session id = 96a464f56b6248c99f4cc594c018d2da-00000000000000000000000000000000
I20260808 10:39:57.808703 8462 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:39:57.810148 9062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:39:57.812518 9062 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: tablet copy: Starting download of 0 data blocks...
I20260808 10:39:57.812732 9062 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: tablet copy: Starting download of 1 WAL segments...
I20260808 10:39:57.814301 9062 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:39:57.816123 8462 tablet_copy_service.cc:342] P 643614d186614f9ea5572a49969ba2e9: Request end of tablet copy session 96a464f56b6248c99f4cc594c018d2da-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.194.124@KRBTEST.COM'} at 127.0.0.1:43620
I20260808 10:39:57.816212 8462 tablet_copy_service.cc:433] P 643614d186614f9ea5572a49969ba2e9: ending tablet copy session 96a464f56b6248c99f4cc594c018d2da-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 96a464f56b6248c99f4cc594c018d2da
I20260808 10:39:57.823468 9062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:57.825644 9260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.826562 9261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.831454 9263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:57.832146 9062 server_base.cc:1034] running on GCE node
I20260808 10:39:57.832353 9062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:57.832677 9062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:57.833824 9062 hybrid_clock.cc:648] HybridClock initialized: now 1786185597833803 us; error 46 us; skew 500 ppm
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:57.835980 9062 init.cc:377] Logged in from keytab as oryx/127.1.194.124@KRBTEST.COM (short username oryx)
I20260808 10:39:57.836469 9062 webserver.cc:460] Webserver started at http://127.1.194.124:33785/ using document root <none> and password file <none>
I20260808 10:39:57.836676 9062 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.836730 9062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.838227 9062 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:57.839319 9271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.839586 9062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:39:57.839704 9062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
uuid: "96a464f56b6248c99f4cc594c018d2da"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "f929c4d906cdc122b769fd4af2cee498"
server_key_iv: "3a574abf26b1ea28ea0d753d60eb82a4"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.839854 9062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.850390 8792 heartbeater.cc:503] Master 127.1.194.126:42389 was elected leader, sending a full tablet report...
I20260808 10:39:57.853451 9062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:57.856629 9062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:57.862001 9062 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.124:46267
I20260808 10:39:57.862036 9329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.124:46267 every 8 connection(s)
I20260808 10:39:57.862661 9062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
I20260808 10:39:57.864470 9330 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:39:57.865634 9330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Bootstrap starting.
W20260808 10:39:57.875741 9248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:57.875986 9248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:57.876073 9248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:39:57.877292 9330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:39:57.877622 9330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: Bootstrap complete.
I20260808 10:39:57.878221 9330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } }
I20260808 10:39:57.878369 9330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 96a464f56b6248c99f4cc594c018d2da, State: Initialized, Role: LEARNER
I20260808 10:39:57.878476 9330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } }
I20260808 10:39:57.878844 9336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:57.878947 9330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:57.879037 9336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: This master's current role is: LEARNER
I20260808 10:39:57.882017 9347 catalog_manager.cc:1259] Loaded cluster ID: ed16ecec505c4e54809e973b9d4b105b
I20260808 10:39:57.882210 9347 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: loading cluster ID for follower catalog manager: success
W20260808 10:39:57.882272 9248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:57.882345 9248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:57.882452 9248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:39:57.884127 9347 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: acquiring CA information for follower catalog manager: success
I20260808 10:39:57.885250 9347 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:39:57.890894 9248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:40449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=40051
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:57.892160 9248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:57.893429 9248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:57.904332 9352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.904347 9350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:57.904359 9349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:57.904740 9248 server_base.cc:1034] running on GCE node
I20260808 10:39:57.905153 9248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:57.905859 9248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:57.907045 9248 hybrid_clock.cc:648] HybridClock initialized: now 1786185597907017 us; error 52 us; skew 500 ppm
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:57.909971 9248 init.cc:377] Logged in from keytab as oryx/127.1.194.65@KRBTEST.COM (short username oryx)
I20260808 10:39:57.911164 9248 webserver.cc:460] Webserver started at http://127.1.194.65:40051/ using document root <none> and password file <none>
I20260808 10:39:57.911819 9248 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:57.911897 9248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:57.917083 9248 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 10:39:57.919788 9360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.921470 9248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:39:57.921662 9248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "90ffd8c5865d476eb0c12d3ebc1895da"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "9f6103eab36ebadf9d14825ce3288220"
server_key_iv: "47e470fc179bc54596c71d74cd84f1d2"
server_key_version: "encryptionkey@0"
I20260808 10:39:57.922508 9248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:57.942286 9248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:57.945819 9248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:57.945979 9248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:57.946516 9248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:57.947417 9248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:57.947492 9248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.947561 9248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:57.947611 9248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:57.959457 9248 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:40449
I20260808 10:39:57.959507 9483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:40449 every 8 connection(s)
I20260808 10:39:57.960489 9248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:39:57.962109 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 9248
I20260808 10:39:57.962304 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8651
I20260808 10:39:57.970999 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:39121
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:42389
--webserver_port=42847
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.124@KRBTEST.COM
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
Aug 08 10:39:57 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185597, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:39:57.983762 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.965431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:35967 (local address 127.1.194.126:42389)
0808 10:39:57.965619 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.965625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.966235 (+ 610us) server_negotiation.cc:407] Connection header received
0808 10:39:57.966481 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.966487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.966559 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.966652 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.968592 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.969578 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.971820 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.972202 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.972293 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.978294 (+ 6001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.978301 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.978306 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.978338 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.980730 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.981429 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.981436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.981442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.981595 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.981983 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.981989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.981993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.982326 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.982414 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.983370 (+ 956us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.983523 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 10:39:57.984206 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.963728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:54949 (local address 127.1.194.125:40471)
0808 10:39:57.963895 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.963901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.964648 (+ 747us) server_negotiation.cc:407] Connection header received
0808 10:39:57.966626 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.966634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.966700 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.966765 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.968654 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.969584 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.971811 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.972202 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.972293 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.978184 (+ 5891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.978188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.978193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.978233 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.980568 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.982481 (+ 1913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.982485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.982487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.982626 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.983037 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.983041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.983043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.983305 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.983398 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.983848 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.984016 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:39:57.984818 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.963035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:42039 (local address 127.1.194.124:46267)
0808 10:39:57.963446 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:57.963468 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:57.965670 (+ 2202us) server_negotiation.cc:407] Connection header received
0808 10:39:57.966256 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:57.966263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:57.966330 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:57.966415 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:57.968555 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.969805 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.971772 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.972201 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.972312 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.976670 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:57.976676 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:57.976679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:57.976718 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:57.978889 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.980826 (+ 1937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.980830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.980833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.980953 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:57.981557 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:57.981561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:57.981564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:57.981848 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:57.981990 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:57.984187 (+ 2197us) server_negotiation.cc:299] Negotiation successful
0808 10:39:57.984619 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
I20260808 10:39:57.985232 9492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.965005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:35967)
0808 10:39:57.966101 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.966120 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.966314 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.966814 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.966817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.967458 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.968381 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.968404 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.969825 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.969829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.971668 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.971676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.972504 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.972508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.972922 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.974091 (+ 1169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.974110 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.978106 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.980849 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.980854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.980857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.981298 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.981717 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.981722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.981728 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.981841 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.982756 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.982761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.983040 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.984019 (+ 979us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.984212 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"mutex_wait_us":542,"spinlock_wait_cycles":122368,"thread_start_us":119,"threads_started":1}
I20260808 10:39:57.985240 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.963078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.1.194.65:54949)
0808 10:39:57.964482 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.964539 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.966481 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.966941 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.966948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.967488 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.968496 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.968505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.969825 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.969829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.971642 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.971655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.972502 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.972507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.972920 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.973991 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.974027 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.977647 (+ 3620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.982040 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.982044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.982046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.982344 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.982779 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.982782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.982784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.982886 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.983466 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.983469 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.983562 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.984446 (+ 884us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.984614 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":422,"spinlock_wait_cycles":77056,"thread_start_us":978,"threads_started":1}
I20260808 10:39:57.986600 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:57.963081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.1.194.65:42039)
0808 10:39:57.965503 (+ 2422us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:57.965521 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:57.965906 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:57.966741 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:57.966753 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:57.967450 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:57.968378 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.968402 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.969944 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.969950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.971627 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:57.971641 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:57.972502 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:57.972509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:57.973078 (+ 569us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:57.973975 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:57.974001 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:57.976446 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:57.979072 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.979082 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.979101 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.979489 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.981114 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:57.981117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:57.981119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:57.981389 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:57.981986 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:57.981994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:57.983037 (+ 1043us) client_negotiation.cc:769] Sending connection context
0808 10:39:57.983282 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 10:39:57.983709 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2276,"mutex_wait_us":604,"spinlock_wait_cycles":34432,"thread_start_us":207,"threads_started":1}
I20260808 10:39:57.986725 9486 heartbeater.cc:347] Connected to a master server at 127.1.194.125:40471
I20260808 10:39:57.987176 9486 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:57.987555 9484 heartbeater.cc:347] Connected to a master server at 127.1.194.124:46267
I20260808 10:39:57.987641 9484 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:57.987772 9486 heartbeater.cc:511] Master 127.1.194.125:40471 requested a full tablet report, sending...
I20260808 10:39:57.987798 9484 heartbeater.cc:511] Master 127.1.194.124:46267 requested a full tablet report, sending...
I20260808 10:39:57.986806 9485 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42389
I20260808 10:39:57.988325 9485 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:57.988548 9485 heartbeater.cc:511] Master 127.1.194.126:42389 requested a full tablet report, sending...
I20260808 10:39:57.989337 9293 ts_manager.cc:194] Registered new tserver with Master: 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449)
I20260808 10:39:57.989462 9093 ts_manager.cc:194] Registered new tserver with Master: 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449)
I20260808 10:39:57.990020 8440 ts_manager.cc:194] Re-registered known tserver with Master: 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449)
I20260808 10:39:57.990914 8440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:35967
I20260808 10:39:58.080798 8936 heartbeater.cc:503] Master 127.1.194.126:42389 was elected leader, sending a full tablet report...
W20260808 10:39:58.091816 9493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:58.092090 9493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:58.092134 9493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:58.096202 9493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:58.096266 9493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:58.096357 9493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:39:58.101723 9493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:39121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=42847
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:58.102866 9493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:58.103870 9493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:58.113636 9499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:58.113842 9493 server_base.cc:1034] running on GCE node
W20260808 10:39:58.113662 9501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:58.114063 9498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:58.114449 9493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:58.115090 9493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:58.116278 9493 hybrid_clock.cc:648] HybridClock initialized: now 1786185598116212 us; error 88 us; skew 500 ppm
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:58.119184 9493 init.cc:377] Logged in from keytab as oryx/127.1.194.66@KRBTEST.COM (short username oryx)
I20260808 10:39:58.120064 9493 webserver.cc:460] Webserver started at http://127.1.194.66:42847/ using document root <none> and password file <none>
I20260808 10:39:58.120630 9493 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:58.120746 9493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:58.125182 9493 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:39:58.127807 9508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:58.129305 9493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:39:58.129446 9493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "82f336cdb47444bc92539988dac37388"
format_stamp: "Formatted at 2026-08-08 10:39:56 on dist-test-slave-ss4n"
server_key: "aa2daa51de5c3e1c2f1eb6b59623099d"
server_key_iv: "dcc86fc27bbba806dc003b18cbb0a23c"
server_key_version: "encryptionkey@0"
I20260808 10:39:58.130072 9493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:58.141593 9493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:58.144884 9493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:58.145118 9493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:58.145829 9493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:58.146701 9493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:58.146772 9493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:58.146843 9493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:58.146894 9493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:58.157275 9493 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:39121
I20260808 10:39:58.157339 9636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:39121 every 8 connection(s)
I20260808 10:39:58.158357 9493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:39:58.159312 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 9493
I20260808 10:39:58.159476 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8797
I20260808 10:39:58.169957 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:39981
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:42389
--webserver_port=44921
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.124@KRBTEST.COM
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:39:58.179320 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.161983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:51927 (local address 127.1.194.126:42389)
0808 10:39:58.162185 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.162192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.164776 (+ 2584us) server_negotiation.cc:407] Connection header received
0808 10:39:58.164955 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.164958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.165011 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.165085 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.166532 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.167347 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.169587 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.169932 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.170008 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.174559 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.174562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.174565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.174593 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.176657 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177345 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.177351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.177353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.177447 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177886 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.177889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.177890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.178121 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.178207 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.178999 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.179133 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:39:58.179320 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.163506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:45741 (local address 127.1.194.124:46267)
0808 10:39:58.163641 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.163646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.164095 (+ 449us) server_negotiation.cc:407] Connection header received
0808 10:39:58.164543 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.164547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.164612 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.164704 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.166516 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.167455 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.169592 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.169949 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.170039 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.174550 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.174554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.174557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.174585 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.176943 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177514 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.177519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.177521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.177611 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177899 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.177902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.177904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.178121 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.178207 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.178999 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.179140 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:39:58.179563 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.161991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:41195 (local address 127.1.194.125:40471)
0808 10:39:58.162147 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.162153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.165553 (+ 3400us) server_negotiation.cc:407] Connection header received
0808 10:39:58.165610 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.165613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.165663 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.165738 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.166521 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.167347 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.169581 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.169932 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.170008 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.174388 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.174395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.174401 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.174439 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.177095 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177690 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.177695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.177699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.177795 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.178160 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.178166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.178168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.178544 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.178641 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.179067 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.179226 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:39:58.181197 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.160664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.1.194.66:45741)
0808 10:39:58.163956 (+ 3292us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.163973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.164270 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.164903 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.164915 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.165534 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:58.166347 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.166366 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.167613 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.167619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.169409 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.169424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.170235 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.170241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.170520 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.171618 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:58.171638 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:58.174400 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:58.177075 (+ 2675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.177081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.177381 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.177718 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.177723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.177796 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.178281 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:58.178288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:58.178690 (+ 402us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.180111 (+ 1421us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.180241 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3186,"mutex_wait_us":436,"spinlock_wait_cycles":59008,"thread_start_us":220,"threads_started":1}
I20260808 10:39:58.181190 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.160664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.1.194.66:41195)
0808 10:39:58.162536 (+ 1872us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.162609 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.164257 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.165851 (+ 1594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.165854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.166031 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:58.166343 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.166363 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.167539 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.167543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.169423 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.169430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.170550 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.170553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.170634 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.171486 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:58.171499 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:58.174198 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:58.177239 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.177247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.177561 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.177919 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.177929 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.178029 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.178711 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:58.178717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:58.178834 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.179773 (+ 939us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.180083 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1603,"mutex_wait_us":336,"spinlock_wait_cycles":9984,"thread_start_us":818,"threads_started":1}
I20260808 10:39:58.181190 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.161061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:51927)
0808 10:39:58.164699 (+ 3638us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.164712 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.164836 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.165205 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.165209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.165545 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:58.166343 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.166363 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.167539 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.167543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.169412 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.169423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.170230 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.170235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.170514 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.171387 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:58.171421 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:58.174400 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:58.176800 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.176809 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.176826 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.177223 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.177594 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.177597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.177600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.177771 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.178281 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:58.178288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:58.178690 (+ 402us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.179541 (+ 851us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.179954 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3548,"mutex_wait_us":527,"spinlock_wait_cycles":40704,"thread_start_us":113,"threads_started":1}
I20260808 10:39:58.182566 9638 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42389
I20260808 10:39:58.182567 9639 heartbeater.cc:347] Connected to a master server at 127.1.194.125:40471
I20260808 10:39:58.182824 9637 heartbeater.cc:347] Connected to a master server at 127.1.194.124:46267
I20260808 10:39:58.182832 9639 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:58.182832 9638 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:58.182943 9637 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:58.183506 9637 heartbeater.cc:511] Master 127.1.194.124:46267 requested a full tablet report, sending...
I20260808 10:39:58.183490 9638 heartbeater.cc:511] Master 127.1.194.126:42389 requested a full tablet report, sending...
I20260808 10:39:58.183490 9639 heartbeater.cc:511] Master 127.1.194.125:40471 requested a full tablet report, sending...
I20260808 10:39:58.184635 9093 ts_manager.cc:194] Registered new tserver with Master: 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:39:58.184635 9293 ts_manager.cc:194] Registered new tserver with Master: 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:39:58.184890 8440 ts_manager.cc:194] Re-registered known tserver with Master: 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:39:58.185747 8440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:51927
I20260808 10:39:58.244256 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.240140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50164)
0808 10:39:58.240276 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.240293 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.240444 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.240824 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.240827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.240847 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:58.240997 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.241004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.242119 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.242122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.242925 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.242934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.243905 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.243908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.243980 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.244002 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.244071 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.244132 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:39:58.244320 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.240209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.1.194.124:46267)
0808 10:39:58.240338 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.240345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.240369 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:58.240626 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.240629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.240710 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.240807 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:58.241160 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.241991 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.243066 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.243767 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.243881 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.244066 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.244105 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.244181 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:39:58.245142 9303 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 10:39:58.245587 9336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 643614d186614f9ea5572a49969ba2e9. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:58.245764 9336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: This master's current role is: LEARNER
W20260808 10:39:58.291922 9644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:58.292187 9644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:58.292230 9644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:58.296241 9644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:58.296312 9644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:58.296373 9644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:39:58.302347 9644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:39981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=44921
--tserver_master_addrs=127.1.194.126:42389,127.1.194.125:40471,127.1.194.124:46267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:58.303431 9644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:58.304486 9644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:58.315142 9649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:58.315119 9650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:58.315119 9652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:58.315512 9644 server_base.cc:1034] running on GCE node
I20260808 10:39:58.315956 9644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:58.316589 9644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:58.317785 9644 hybrid_clock.cc:648] HybridClock initialized: now 1786185598317765 us; error 36 us; skew 500 ppm
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:58.320724 9644 init.cc:377] Logged in from keytab as oryx/127.1.194.67@KRBTEST.COM (short username oryx)
I20260808 10:39:58.321664 9644 webserver.cc:460] Webserver started at http://127.1.194.67:44921/ using document root <none> and password file <none>
I20260808 10:39:58.322220 9644 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:58.322293 9644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:58.326400 9644 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 10:39:58.329012 9659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:58.330468 9644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:39:58.330570 9644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "2cb41e7dd76845ba8af80e6db0b66d68"
format_stamp: "Formatted at 2026-08-08 10:39:57 on dist-test-slave-ss4n"
server_key: "6b6a4e6b60eb5c306a36556f84c3f114"
server_key_iv: "40f8ef88dd4e4c59daa3d770d670fe6e"
server_key_version: "encryptionkey@0"
I20260808 10:39:58.331130 9644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:58.353209 9644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:58.356729 9644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:58.356899 9644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:58.357576 9644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:39:58.358541 9644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:39:58.358603 9644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:58.358646 9644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:39:58.358670 9644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:58.368613 9644 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:39981
I20260808 10:39:58.368651 9778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:39981 every 8 connection(s)
I20260808 10:39:58.369771 9644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:39:58.378434 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 9644
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.124@KRBTEST.COM
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.125@KRBTEST.COM
Aug 08 10:39:58 dist-test-slave-ss4n krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185598, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:39:58.388278 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.371689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:39645 (local address 127.1.194.125:40471)
0808 10:39:58.371978 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.371982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.373824 (+ 1842us) server_negotiation.cc:407] Connection header received
0808 10:39:58.374228 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.374231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.374286 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.374342 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.375849 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.376689 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378102 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.378482 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378579 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.383447 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.383451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.383454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.383485 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.385411 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.386155 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.386158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.386161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.386258 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.386789 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.386794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.386796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.387075 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.387175 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.387950 (+ 775us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.388090 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260808 10:39:58.389058 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.371838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:59791 (local address 127.1.194.126:42389)
0808 10:39:58.372097 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.372101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.373537 (+ 1436us) server_negotiation.cc:407] Connection header received
0808 10:39:58.374226 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.374229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.374295 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.374379 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.375824 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.376521 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378018 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.378378 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378502 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.383696 (+ 5194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.383700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.383705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.383744 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.386014 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.387270 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.387274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.387276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.387384 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.387729 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.387732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.387734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.387976 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.388066 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.388762 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.388891 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260808 10:39:58.390249 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.372198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.1.194.67:39645)
0808 10:39:58.373707 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.373721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.373994 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.374565 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.374575 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.375088 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:58.375682 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.375691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.376810 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.376813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.377924 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.377932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.378727 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.378958 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.379951 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:58.380008 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:58.383279 (+ 3271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:58.385590 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.385599 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.385617 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.386026 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.386386 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.386391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.386397 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.386642 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.387251 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:58.387259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:58.387630 (+ 371us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.388526 (+ 896us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.388914 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1412,"mutex_wait_us":392,"spinlock_wait_cycles":1292416,"thread_start_us":701,"threads_started":1}
I20260808 10:39:58.390249 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.372197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.67:59791)
0808 10:39:58.373407 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.373421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.373997 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.374568 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.374578 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.375083 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:58.375670 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.375686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.376694 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.376697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.377844 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.377855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.378702 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.378959 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.380098 (+ 1139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:58.380116 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:58.383167 (+ 3051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:58.386150 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.386153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.386156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.387143 (+ 987us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.387514 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.387517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.387519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.387609 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.388130 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:58.388133 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:58.388540 (+ 407us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.389206 (+ 666us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.389393 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"mutex_wait_us":212,"spinlock_wait_cycles":65920,"thread_start_us":798,"threads_started":1}
I20260808 10:39:58.391674 9780 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42389
I20260808 10:39:58.391724 9781 heartbeater.cc:347] Connected to a master server at 127.1.194.125:40471
I20260808 10:39:58.391898 9780 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:58.391901 9781 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:58.392618 9780 heartbeater.cc:511] Master 127.1.194.126:42389 requested a full tablet report, sending...
I20260808 10:39:58.392618 9781 heartbeater.cc:511] Master 127.1.194.125:40471 requested a full tablet report, sending...
I20260808 10:39:58.392792 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.372301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:36299 (local address 127.1.194.124:46267)
0808 10:39:58.372441 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.372445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.372966 (+ 521us) server_negotiation.cc:407] Connection header received
0808 10:39:58.374228 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.374231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.374290 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.374342 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.375842 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.376657 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378079 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.378418 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378506 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.383805 (+ 5299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.383810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.383813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.383842 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.385875 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.388633 (+ 2758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.388636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.388638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.388729 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.389681 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.389686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.389690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.389942 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.390033 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.392447 (+ 2414us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.392596 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:39:58.393536 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.372197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.1.194.67:36299)
0808 10:39:58.372795 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.372851 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.373994 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.374565 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.374575 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.375088 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:58.375685 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.375691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.376810 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.376813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.377922 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.377932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.378701 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.378704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.378949 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.379686 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:58.379711 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:58.383413 (+ 3702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:58.386578 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.386582 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.386585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.386841 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.389387 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:58.389393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:58.389396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:58.389536 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:58.390096 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:58.390099 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:58.390193 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.393126 (+ 2933us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.393283 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":245,"spinlock_wait_cycles":28288,"thread_start_us":167,"threads_started":1}
I20260808 10:39:58.393851 8440 ts_manager.cc:194] Re-registered known tserver with Master: 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981)
I20260808 10:39:58.393965 9093 ts_manager.cc:194] Registered new tserver with Master: 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981)
I20260808 10:39:58.394598 9779 heartbeater.cc:347] Connected to a master server at 127.1.194.124:46267
I20260808 10:39:58.394618 8440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:59791
I20260808 10:39:58.394686 9779 heartbeater.cc:464] Registering TS with master...
I20260808 10:39:58.394842 9779 heartbeater.cc:511] Master 127.1.194.124:46267 requested a full tablet report, sending...
I20260808 10:39:58.395519 9293 ts_manager.cc:194] Registered new tserver with Master: 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981)
I20260808 10:39:58.397831 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.381907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.1.194.125:40471)
0808 10:39:58.382186 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.382189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.383299 (+ 1110us) server_negotiation.cc:407] Connection header received
0808 10:39:58.383507 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.383513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.383575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.383645 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.385178 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.386059 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.387280 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.387651 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.387742 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.391138 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.391145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.391150 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.391182 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.393216 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.393900 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.393903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.393904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.393996 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.396727 (+ 2731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.396731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.396733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.396978 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.397088 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.397525 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.397652 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":109,"threads_started":1}
I20260808 10:39:58.400905 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.383199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.1.194.126:42389)
0808 10:39:58.385033 (+ 1834us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.385074 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.387207 (+ 2133us) server_negotiation.cc:407] Connection header received
0808 10:39:58.387256 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.387261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.387312 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.387387 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.391222 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.392233 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.395031 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.395354 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.395449 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.397205 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.397209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.397212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.397236 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.399196 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.399646 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.399650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.399652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.399744 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.400050 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.400054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.400056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.400241 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.400388 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.400611 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.400710 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1750,"thread_start_us":89,"threads_started":1}
I20260808 10:39:58.402348 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.382532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.1.194.124:46267)
0808 10:39:58.382781 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.382786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.386310 (+ 3524us) server_negotiation.cc:407] Connection header received
0808 10:39:58.386350 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.386352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.386400 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.386477 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.388320 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.389129 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.396589 (+ 7460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.396859 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.396991 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.398741 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.398744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.398747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.398771 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.400378 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.400874 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.400878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.400880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.400969 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.401326 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.401329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.401330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.401559 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.401717 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.402001 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.402144 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260808 10:39:58.404083 8440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:39:58.407272 8440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:39:58.409852 9798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: NON_VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: true } } }
I20260808 10:39:58.409986 9798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:58.413254 9798 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: attempting to promote NON_VOTER 96a464f56b6248c99f4cc594c018d2da to VOTER
I20260808 10:39:58.413594 9798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } }
I20260808 10:39:58.417119 9303 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 LEARNER]: Refusing update from remote peer 643614d186614f9ea5572a49969ba2e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:39:58.417192 9102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Refusing update from remote peer 643614d186614f9ea5572a49969ba2e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260808 10:39:58.417635 9797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:39:58.417809 9047 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:39:58.419762 9047 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 96a464f56b6248c99f4cc594c018d2da (127.1.194.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.420434 9102 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 96a464f56b6248c99f4cc594c018d2da (127.1.194.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.421499 9047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.421627 9047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:58.421802 9797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.421862 9047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.421970 9797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:58.421995 9047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:58.422845 9303 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 96a464f56b6248c99f4cc594c018d2da (127.1.194.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.425029 9336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.425158 9336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:58.427163 9807 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "643614d186614f9ea5572a49969ba2e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:39:58.427402 9807 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:58.430176 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.423825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40449 (local address 127.0.0.1:55608)
0808 10:39:58.423983 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.423999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.424093 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.425734 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.425738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.425757 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:58.425877 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.425886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.427457 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.427461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.428562 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.428573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.429793 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.429797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.429886 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.429913 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.429959 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.430043 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 10:39:58.430509 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.424449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:39121 (local address 127.0.0.1:38954)
0808 10:39:58.425985 (+ 1536us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.426000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.426081 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.426900 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.426903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.426921 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:58.427019 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.427027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.428290 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.428295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.429273 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.429283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.430127 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.430131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.430217 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.430238 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.430299 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.430371 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1462,"thread_start_us":114,"threads_started":1}
I20260808 10:39:58.430953 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.424010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.1.194.65:40449)
0808 10:39:58.424473 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.424495 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.424522 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:39:58.424607 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.424613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.425536 (+ 923us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.425769 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:58.426040 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.427291 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.428749 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.429657 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.430438 (+ 781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.430680 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.430728 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.430816 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":92,"threads_started":1}
I20260808 10:39:58.430982 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.425039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38954 (local address 127.1.194.66:39121)
0808 10:39:58.426436 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.426447 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.426474 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:39:58.426538 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.426542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.426753 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.426924 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:58.427177 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.428158 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.429401 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.429996 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.430134 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.430297 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.430470 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.430871 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1292,"thread_start_us":136,"threads_started":1}
I20260808 10:39:58.431969 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.424699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:39981 (local address 127.0.0.1:46572)
0808 10:39:58.426255 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.426267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.426353 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.427038 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.427041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.427059 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:58.427143 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.427148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.429327 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.429330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.430564 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.430576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.431558 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.431562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.431657 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.431684 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.431769 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.431843 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1503,"thread_start_us":96,"threads_started":1}
I20260808 10:39:58.432003 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.424874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.1.194.67:39981)
0808 10:39:58.426081 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.426095 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.426447 (+ 352us) server_negotiation.cc:407] Connection header received
0808 10:39:58.426540 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.426545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.426743 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.426896 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:58.427258 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.429198 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.430713 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.431436 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.431532 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.431695 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.431806 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.431897 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1092,"thread_start_us":108,"threads_started":1}
I20260808 10:39:58.434831 9561 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2b99ee45b84935bb14b37faa3ef93c (DEFAULT_TABLE table=test-table [id=786b4d62b6c446248defb47e424001ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:58.435467 9416 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2b99ee45b84935bb14b37faa3ef93c (DEFAULT_TABLE table=test-table [id=786b4d62b6c446248defb47e424001ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:58.436046 9561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2b99ee45b84935bb14b37faa3ef93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:58.436434 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2b99ee45b84935bb14b37faa3ef93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:58.440303 9711 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2b99ee45b84935bb14b37faa3ef93c (DEFAULT_TABLE table=test-table [id=786b4d62b6c446248defb47e424001ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:39:58.441349 9711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2b99ee45b84935bb14b37faa3ef93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:58.444706 9813 tablet_bootstrap.cc:492] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Bootstrap starting.
I20260808 10:39:58.447186 9813 tablet_bootstrap.cc:654] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:58.448184 9814 tablet_bootstrap.cc:492] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Bootstrap starting.
I20260808 10:39:58.448186 9813 log.cc:824] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:58.449008 9815 tablet_bootstrap.cc:492] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Bootstrap starting.
I20260808 10:39:58.450520 9814 tablet_bootstrap.cc:654] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:58.450856 9813 tablet_bootstrap.cc:492] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: No bootstrap required, opened a new log
I20260808 10:39:58.451062 9815 tablet_bootstrap.cc:654] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:58.451081 9813 ts_tablet_manager.cc:1397] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:39:58.451390 9814 log.cc:824] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:58.451857 9815 log.cc:824] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:58.453325 9814 tablet_bootstrap.cc:492] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: No bootstrap required, opened a new log
I20260808 10:39:58.453580 9814 ts_tablet_manager.cc:1397] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:39:58.453868 9815 tablet_bootstrap.cc:492] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: No bootstrap required, opened a new log
I20260808 10:39:58.454077 9815 ts_tablet_manager.cc:1397] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:39:58.456058 9813 raft_consensus.cc:348] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.456349 9813 raft_consensus.cc:374] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:58.456408 9813 raft_consensus.cc:729] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f336cdb47444bc92539988dac37388, State: Initialized, Role: FOLLOWER
I20260808 10:39:58.456877 9813 consensus_queue.cc:260] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.457695 9814 raft_consensus.cc:348] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.457937 9814 raft_consensus.cc:374] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:58.457988 9814 raft_consensus.cc:729] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ffd8c5865d476eb0c12d3ebc1895da, State: Initialized, Role: FOLLOWER
I20260808 10:39:58.458096 9813 ts_tablet_manager.cc:1428] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 10:39:58.458288 9638 heartbeater.cc:503] Master 127.1.194.126:42389 was elected leader, sending a full tablet report...
I20260808 10:39:58.458422 9814 consensus_queue.cc:260] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.459069 9815 raft_consensus.cc:348] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.459326 9815 raft_consensus.cc:374] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:58.459362 9814 ts_tablet_manager.cc:1428] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:39:58.459376 9815 raft_consensus.cc:729] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb41e7dd76845ba8af80e6db0b66d68, State: Initialized, Role: FOLLOWER
I20260808 10:39:58.459780 9815 consensus_queue.cc:260] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.460676 9485 heartbeater.cc:503] Master 127.1.194.126:42389 was elected leader, sending a full tablet report...
I20260808 10:39:58.460703 9815 ts_tablet_manager.cc:1428] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:39:58.461777 9780 heartbeater.cc:503] Master 127.1.194.126:42389 was elected leader, sending a full tablet report...
W20260808 10:39:58.463637 9487 tablet.cc:2369] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:58.621822 9782 tablet.cc:2369] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:58.665783 9640 tablet.cc:2369] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:58.775892 9822 raft_consensus.cc:482] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:39:58.776131 9822 raft_consensus.cc:504] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.777441 9822 leader_election.cc:288] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449), 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:39:58.782490 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.777876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:52277 (local address 127.1.194.66:39121)
0808 10:39:58.778017 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.778022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.778117 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:39:58.778273 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.778275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.778346 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.778426 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:58.778782 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.779475 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.781013 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.781733 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.781864 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.781935 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.782253 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.782329 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:39:58.782531 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.777791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40449 (local address 127.1.194.67:60477)
0808 10:39:58.778002 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.778018 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.778122 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.778367 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.778372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.778392 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:58.778540 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.778553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.779464 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.779467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.780964 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.780979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.781958 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.781960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.782099 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.782124 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.782226 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.782384 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":112}
I20260808 10:39:58.782588 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.777869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:60477 (local address 127.1.194.65:40449)
0808 10:39:58.778125 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.778130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.778147 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:39:58.778210 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.778213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.778270 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.778329 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:39:58.778672 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.779349 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.781159 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.781850 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.781973 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.782015 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.782368 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.782453 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260808 10:39:58.782653 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.777786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:39121 (local address 127.1.194.67:52277)
0808 10:39:58.778005 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:58.778023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:58.778142 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:58.778469 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:58.778472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:58.778536 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:39:58.778664 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.778671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.779587 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.779589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.780870 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:58.780882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.781856 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.781858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:58.782060 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.782087 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:39:58.782212 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 10:39:58.782350 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":138}
I20260808 10:39:58.783380 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2b99ee45b84935bb14b37faa3ef93c" candidate_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" is_pre_election: true
I20260808 10:39:58.783551 9583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2b99ee45b84935bb14b37faa3ef93c" candidate_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f336cdb47444bc92539988dac37388" is_pre_election: true
I20260808 10:39:58.783767 9437 raft_consensus.cc:2454] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cb41e7dd76845ba8af80e6db0b66d68 in term 0.
I20260808 10:39:58.783788 9583 raft_consensus.cc:2454] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cb41e7dd76845ba8af80e6db0b66d68 in term 0.
I20260808 10:39:58.784314 9661 leader_election.cc:302] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb41e7dd76845ba8af80e6db0b66d68, 90ffd8c5865d476eb0c12d3ebc1895da; no voters:
I20260808 10:39:58.785126 9822 raft_consensus.cc:2781] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:39:58.785225 9822 raft_consensus.cc:482] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:39:58.785264 9822 raft_consensus.cc:3037] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:58.786597 9822 raft_consensus.cc:504] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.786934 9822 leader_election.cc:288] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [CANDIDATE]: Term 1 election: Requested vote from peers 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449), 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:39:58.787286 9583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2b99ee45b84935bb14b37faa3ef93c" candidate_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f336cdb47444bc92539988dac37388"
I20260808 10:39:58.787333 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2b99ee45b84935bb14b37faa3ef93c" candidate_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ffd8c5865d476eb0c12d3ebc1895da"
I20260808 10:39:58.787429 9583 raft_consensus.cc:3037] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:58.787473 9437 raft_consensus.cc:3037] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:58.788425 9583 raft_consensus.cc:2454] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cb41e7dd76845ba8af80e6db0b66d68 in term 1.
I20260808 10:39:58.788694 9437 raft_consensus.cc:2454] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cb41e7dd76845ba8af80e6db0b66d68 in term 1.
I20260808 10:39:58.788719 9663 leader_election.cc:302] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb41e7dd76845ba8af80e6db0b66d68, 82f336cdb47444bc92539988dac37388; no voters:
I20260808 10:39:58.788900 9822 raft_consensus.cc:2781] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:58.789114 9822 raft_consensus.cc:686] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 LEADER]: Becoming Leader. State: Replica: 2cb41e7dd76845ba8af80e6db0b66d68, State: Running, Role: LEADER
I20260808 10:39:58.789568 9822 consensus_queue.cc:237] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:39:58.793020 8440 catalog_manager.cc:5495] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } health_report { overall_health: UNKNOWN } } }
I20260808 10:39:58.881976 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.878151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.1.194.67:39981)
0808 10:39:58.878307 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.878313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.878333 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:39:58.878476 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.878478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.878534 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.878628 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:58.878853 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.879482 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.880443 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.880666 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.880798 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.881141 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:58.881570 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:58.881697 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.881765 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.881821 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:39:58.887928 9437 raft_consensus.cc:1261] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 FOLLOWER]: Refusing update from remote peer 2cb41e7dd76845ba8af80e6db0b66d68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:58.887939 9583 raft_consensus.cc:1261] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Refusing update from remote peer 2cb41e7dd76845ba8af80e6db0b66d68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:58.888841 9823 consensus_queue.cc:1035] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:58.889097 9822 consensus_queue.cc:1035] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:58.908828 8437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:58.909025 8437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:39:58.913511 8437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 643614d186614f9ea5572a49969ba2e9: Sending DeleteTablet for 3 replicas of tablet 7b2b99ee45b84935bb14b37faa3ef93c
I20260808 10:39:58.914544 9416 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2b99ee45b84935bb14b37faa3ef93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:58 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:55608
I20260808 10:39:58.914594 9561 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2b99ee45b84935bb14b37faa3ef93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:58 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:38954
I20260808 10:39:58.914633 9711 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2b99ee45b84935bb14b37faa3ef93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:39:58 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:46572
I20260808 10:39:58.915087 9841 tablet_replica.cc:330] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: stopping tablet replica
I20260808 10:39:58.915406 9841 raft_consensus.cc:2229] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:39:58.915743 9841 raft_consensus.cc:2258] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:58.917452 9841 ts_tablet_manager.cc:1905] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:58.917826 9840 tablet_replica.cc:330] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: stopping tablet replica
I20260808 10:39:58.918169 9840 raft_consensus.cc:2229] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:58.918479 9840 raft_consensus.cc:2258] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:58.919246 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8404
I20260808 10:39:58.920464 9841 ts_tablet_manager.cc:1918] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:58.920583 9841 log.cc:1196] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/7b2b99ee45b84935bb14b37faa3ef93c
I20260808 10:39:58.920872 9841 ts_tablet_manager.cc:1939] T 7b2b99ee45b84935bb14b37faa3ef93c P 2cb41e7dd76845ba8af80e6db0b66d68: Deleting consensus metadata
I20260808 10:39:58.926214 9839 tablet_replica.cc:330] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: stopping tablet replica
I20260808 10:39:58.927003 9840 ts_tablet_manager.cc:1905] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:39:58.927786 9661 connection.cc:535] server connection from 127.0.0.1:46572 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:39:58.929155 9839 raft_consensus.cc:2229] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:39:58.929551 9839 raft_consensus.cc:2258] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:39:58.930207 9840 ts_tablet_manager.cc:1918] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:58.930342 9840 log.cc:1196] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/7b2b99ee45b84935bb14b37faa3ef93c
I20260808 10:39:58.930771 9840 ts_tablet_manager.cc:1939] T 7b2b99ee45b84935bb14b37faa3ef93c P 82f336cdb47444bc92539988dac37388: Deleting consensus metadata
W20260808 10:39:58.931859 9513 connection.cc:441] server connection from 127.0.0.1:38954 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38954 (request call id 1) could send its response
I20260808 10:39:58.934788 9839 ts_tablet_manager.cc:1905] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:39:58.938123 9839 ts_tablet_manager.cc:1918] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:39:58.938323 9839 log.cc:1196] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/7b2b99ee45b84935bb14b37faa3ef93c
I20260808 10:39:58.938791 9839 ts_tablet_manager.cc:1939] T 7b2b99ee45b84935bb14b37faa3ef93c P 90ffd8c5865d476eb0c12d3ebc1895da: Deleting consensus metadata
W20260808 10:39:58.940263 9362 connection.cc:441] server connection from 127.0.0.1:55608 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55608 (request call id 1) could send its response
I20260808 10:39:58.943949 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.932259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.1.194.125:40471)
0808 10:39:58.932586 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.932591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.932789 (+ 198us) server_negotiation.cc:407] Connection header received
0808 10:39:58.932947 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.932950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.933008 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.933094 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.934287 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.935128 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.936242 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.936564 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.936676 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.939019 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.939026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.939032 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.939062 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.941271 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.941942 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.941945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.941947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.942036 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.942344 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.942348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.942350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.942633 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.942743 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.943448 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.943592 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":62,"threads_started":1}
I20260808 10:39:58.945300 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.917596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.1.194.125:40471)
0808 10:39:58.926664 (+ 9068us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.926669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.926686 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:39:58.926733 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.926736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.926787 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.926881 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.928095 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.932490 (+ 4395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.933729 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.934084 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.934180 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.936220 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.936224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.936227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.936253 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.938858 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.941433 (+ 2575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.941436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.941438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.941688 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.942020 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.942024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.942026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.942277 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.942397 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.945017 (+ 2620us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.945148 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8979,"thread_start_us":76,"threads_started":1}
I20260808 10:39:58.949271 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.937449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.1.194.124:46267)
0808 10:39:58.939494 (+ 2045us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.939498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.939512 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:39:58.939555 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.939557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.939603 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.940455 (+ 852us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.940594 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.941348 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.942558 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.942891 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.943357 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.944681 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.944685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.944688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.944712 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.946932 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.947404 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.947409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.947412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.947523 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.947859 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.947865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.947870 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.948122 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.948264 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.948937 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.949096 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1966,"spinlock_wait_cycles":3968,"thread_start_us":122,"threads_started":1}
I20260808 10:39:58.952651 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:58.917793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.1.194.124:46267)
0808 10:39:58.931863 (+ 14070us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:58.931868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:58.943161 (+ 11293us) server_negotiation.cc:407] Connection header received
0808 10:39:58.943200 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:58.943203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:58.943251 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:58.943613 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:58.944455 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.945519 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.946521 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:58.946759 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:58.946871 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:58.948457 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:58.948461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:58.948463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:58.948486 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:58.950239 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.950682 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.950685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.950686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.950758 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:58.951021 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:58.951023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:58.951024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:58.951178 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:58.951296 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:58.952355 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0808 10:39:58.952475 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13986,"spinlock_wait_cycles":2944,"thread_start_us":65,"threads_started":1}
I20260808 10:39:59.904304 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:59.903732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:44559)
0808 10:39:59.904070 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:59.904181 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":117,"threads_started":1}
I20260808 10:39:59.904458 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:59.903940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:58739)
0808 10:39:59.904281 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:59.904363 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":99,"threads_started":1}
W20260808 10:39:59.904758 9638 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:42389 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
W20260808 10:39:59.904861 9485 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:42389 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:39:59.905058 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:59.904561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.67:60519)
0808 10:39:59.904801 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:59.904947 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":75,"threads_started":1}
W20260808 10:39:59.905362 9780 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:42389 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:40:00.414510 9861 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 643614d186614f9ea5572a49969ba2e9)
I20260808 10:40:00.414808 9861 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } }
I20260808 10:40:00.416205 9861 leader_election.cc:288] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389), 96a464f56b6248c99f4cc594c018d2da (127.1.194.124:46267)
I20260808 10:40:00.416783 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.416346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44118)
0808 10:40:00.416582 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.416674 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
W20260808 10:40:00.417052 9074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:40:00.418028 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.417754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44120)
0808 10:40:00.417890 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.417954 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 10:40:00.418085 9868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 643614d186614f9ea5572a49969ba2e9)
W20260808 10:40:00.418172 9074 leader_election.cc:334] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389): Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:40:00.418255 9868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } }
I20260808 10:40:00.419221 9868 leader_election.cc:288] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389), cd7313f308b646f1af7676abc36a516e (127.1.194.125:40471)
I20260808 10:40:00.419703 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.419322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44130)
0808 10:40:00.419525 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.419613 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":64,"threads_started":1}
W20260808 10:40:00.419862 9272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:40:00.420720 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.420478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44144)
0808 10:40:00.420603 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.420655 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 10:40:00.420821 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.416409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:46267 (local address 127.0.0.1:50194)
0808 10:40:00.416728 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.416743 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.416851 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.417167 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.417170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.417190 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.417300 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.417306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.418352 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.418354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.419465 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.419475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.420381 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.420385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.420489 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.420512 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.420588 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.420675 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
W20260808 10:40:00.420868 9272 leader_election.cc:334] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389): Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:40:00.421181 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.416600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50194 (local address 127.1.194.124:46267)
0808 10:40:00.416846 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.416851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.416866 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:00.417009 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.417012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.417057 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.417154 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.417419 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.418212 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.419602 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.420253 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.420371 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.420504 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.421022 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.421098 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260808 10:40:00.421645 9303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd7313f308b646f1af7676abc36a516e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "96a464f56b6248c99f4cc594c018d2da" is_pre_election: true
I20260808 10:40:00.421808 9303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd7313f308b646f1af7676abc36a516e in term 1.
I20260808 10:40:00.422214 9075 leader_election.cc:302] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a464f56b6248c99f4cc594c018d2da, cd7313f308b646f1af7676abc36a516e; no voters: 643614d186614f9ea5572a49969ba2e9
I20260808 10:40:00.422427 9861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:40:00.422489 9861 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 643614d186614f9ea5572a49969ba2e9)
I20260808 10:40:00.422538 9861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:40:00.423487 9872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.419664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.1.194.125:40471)
0808 10:40:00.419992 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.420000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.420014 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:00.420065 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.420067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.420102 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.420213 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.420606 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.421341 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.422627 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.423023 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.423105 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.423145 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.423351 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.423398 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":104,"threads_started":1}
I20260808 10:40:00.423573 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.419391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:40471 (local address 127.0.0.1:40198)
0808 10:40:00.419600 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.419614 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.419691 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.420323 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.420326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.420345 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.420479 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.420486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.421434 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.421439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.422459 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.422468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.423143 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.423147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.423233 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.423257 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.423342 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.423419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260808 10:40:00.423851 9861 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } }
I20260808 10:40:00.424188 9861 leader_election.cc:288] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [CANDIDATE]: Term 2 election: Requested vote from peers 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389), 96a464f56b6248c99f4cc594c018d2da (127.1.194.124:46267)
I20260808 10:40:00.424582 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.424298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44154)
0808 10:40:00.424386 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.424448 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 10:40:00.424589 9303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd7313f308b646f1af7676abc36a516e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "96a464f56b6248c99f4cc594c018d2da"
I20260808 10:40:00.424708 9303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:40:00.424811 9102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96a464f56b6248c99f4cc594c018d2da" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cd7313f308b646f1af7676abc36a516e" is_pre_election: true
I20260808 10:40:00.425022 9102 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a464f56b6248c99f4cc594c018d2da in current term 2: Already voted for candidate cd7313f308b646f1af7676abc36a516e in this term.
I20260808 10:40:00.425235 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.425014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44160)
0808 10:40:00.425122 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.425176 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:40:00.425318 9074 leader_election.cc:334] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389): Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
I20260808 10:40:00.425418 9276 leader_election.cc:302] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a464f56b6248c99f4cc594c018d2da; no voters: 643614d186614f9ea5572a49969ba2e9, cd7313f308b646f1af7676abc36a516e
I20260808 10:40:00.425915 9303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd7313f308b646f1af7676abc36a516e in term 2.
I20260808 10:40:00.426008 9868 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:40:00.426266 9075 leader_election.cc:302] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a464f56b6248c99f4cc594c018d2da, cd7313f308b646f1af7676abc36a516e; no voters: 643614d186614f9ea5572a49969ba2e9
I20260808 10:40:00.426419 9861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:40:00.426630 9861 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [term 2 LEADER]: Becoming Leader. State: Replica: cd7313f308b646f1af7676abc36a516e, State: Running, Role: LEADER
I20260808 10:40:00.426872 9861 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } }
I20260808 10:40:00.428282 9874 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7313f308b646f1af7676abc36a516e. Latest consensus state: current_term: 2 leader_uuid: "cd7313f308b646f1af7676abc36a516e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:40:00.428421 9874 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:00.428766 9876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:00.431249 9876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:00.431591 9876 catalog_manager.cc:1259] Loaded cluster ID: ed16ecec505c4e54809e973b9d4b105b
I20260808 10:40:00.431645 9876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:00.432135 9876 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:00.432468 9876 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Loaded TSK: 0
I20260808 10:40:00.433072 9876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:40:00.466701 9093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:40:00.469959 9093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:00.473023 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.472784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44190)
0808 10:40:00.472893 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.472948 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 10:40:00.473225 9303 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [term 2 FOLLOWER]: Refusing update from remote peer cd7313f308b646f1af7676abc36a516e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 10:40:00.473730 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.473473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.0.0.1:44198)
0808 10:40:00.473602 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.473655 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 10:40:00.473951 9874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 10:40:00.474141 9074 consensus_peers.cc:487] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e -> Peer 643614d186614f9ea5572a49969ba2e9 (127.1.194.126:42389): Couldn't send request to peer 643614d186614f9ea5572a49969ba2e9. Status: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:40:00.475665 9868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7313f308b646f1af7676abc36a516e. Latest consensus state: current_term: 2 leader_uuid: "cd7313f308b646f1af7676abc36a516e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:40:00.475845 9868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:40:00.477357 9874 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cd7313f308b646f1af7676abc36a516e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:40:00.477753 9868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cd7313f308b646f1af7676abc36a516e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "643614d186614f9ea5572a49969ba2e9" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42389 } } peers { permanent_uuid: "cd7313f308b646f1af7676abc36a516e" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "96a464f56b6248c99f4cc594c018d2da" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 46267 } attrs { promote: false } } }
I20260808 10:40:00.477885 9868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a464f56b6248c99f4cc594c018d2da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:40:00.477730 9874 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:00.488667 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.484218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40449 (local address 127.0.0.1:55612)
0808 10:40:00.484343 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.484360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.484448 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.484985 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.484987 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.485003 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.485115 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.485121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.486304 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.486306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.487399 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.487409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.488281 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.488284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.488373 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.488400 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.488458 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.488531 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:40:00.489219 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.484397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.1.194.65:40449)
0808 10:40:00.484710 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.484716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.484730 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:00.484782 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.484785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.484837 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.484980 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.485248 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.486195 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.487534 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.488154 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.488903 (+ 749us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.488963 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.488997 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.489077 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
I20260808 10:40:00.489398 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.484486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:39981 (local address 127.0.0.1:46588)
0808 10:40:00.484758 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.484773 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.484856 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.485899 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.485905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.485924 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.486070 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.486076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.487138 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.487144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.488083 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.488093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.489018 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.489020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.489092 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.489109 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.489204 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.489272 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196}
I20260808 10:40:00.489822 9416 tablet_service.cc:1467] Processing CreateTablet for tablet e49df5aaae4a4a8aae429cca4eca5e6f (DEFAULT_TABLE table=test-table [id=496a0b0c1aa849b6af6d556a1a494f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:00.490151 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49df5aaae4a4a8aae429cca4eca5e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:00.490412 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.485096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:39121 (local address 127.0.0.1:38966)
0808 10:40:00.486444 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.486459 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.486617 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.486899 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.486902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.486918 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.487002 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.487008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.487945 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.487948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.489138 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.489148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.490015 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.490019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.490126 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.490149 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.490232 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.490302 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"thread_start_us":138,"threads_started":1}
I20260808 10:40:00.490473 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.485547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.1.194.66:39121)
0808 10:40:00.485886 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.485890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.486554 (+ 664us) server_negotiation.cc:407] Connection header received
0808 10:40:00.486727 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.486730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.486783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.486871 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.487111 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.487824 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.489273 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.489867 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.489949 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.490039 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.490264 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.490337 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":79,"threads_started":1}
I20260808 10:40:00.491077 9561 tablet_service.cc:1467] Processing CreateTablet for tablet e49df5aaae4a4a8aae429cca4eca5e6f (DEFAULT_TABLE table=test-table [id=496a0b0c1aa849b6af6d556a1a494f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:00.491353 9561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49df5aaae4a4a8aae429cca4eca5e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:00.493103 9886 tablet_bootstrap.cc:492] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da: Bootstrap starting.
I20260808 10:40:00.493340 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.484773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.1.194.67:39981)
0808 10:40:00.485658 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.485663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.485677 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:00.485727 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.485730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.485773 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.485860 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.486223 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.486997 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.488221 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.488683 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.488827 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.488903 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.489225 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.493205 (+ 3980us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":78,"threads_started":1}
I20260808 10:40:00.493845 9711 tablet_service.cc:1467] Processing CreateTablet for tablet e49df5aaae4a4a8aae429cca4eca5e6f (DEFAULT_TABLE table=test-table [id=496a0b0c1aa849b6af6d556a1a494f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:00.494055 9886 tablet_bootstrap.cc:654] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:00.494120 9711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49df5aaae4a4a8aae429cca4eca5e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:00.494728 9887 tablet_bootstrap.cc:492] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Bootstrap starting.
I20260808 10:40:00.495584 9887 tablet_bootstrap.cc:654] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:00.496999 9887 tablet_bootstrap.cc:492] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: No bootstrap required, opened a new log
I20260808 10:40:00.497128 9887 ts_tablet_manager.cc:1397] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:00.497411 9886 tablet_bootstrap.cc:492] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da: No bootstrap required, opened a new log
I20260808 10:40:00.497608 9886 ts_tablet_manager.cc:1397] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:40:00.497596 9887 raft_consensus.cc:348] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.497722 9887 raft_consensus.cc:374] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:00.497762 9887 raft_consensus.cc:729] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f336cdb47444bc92539988dac37388, State: Initialized, Role: FOLLOWER
I20260808 10:40:00.497887 9887 consensus_queue.cc:260] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.498126 9886 raft_consensus.cc:348] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.498252 9887 ts_tablet_manager.cc:1428] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:40:00.498255 9886 raft_consensus.cc:374] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:00.498301 9886 raft_consensus.cc:729] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ffd8c5865d476eb0c12d3ebc1895da, State: Initialized, Role: FOLLOWER
I20260808 10:40:00.498442 9886 consensus_queue.cc:260] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.498795 9886 ts_tablet_manager.cc:1428] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:00.500383 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.499904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:35597)
0808 10:40:00.500214 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.500278 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260808 10:40:00.501034 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.500819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:45949)
0808 10:40:00.500928 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.500975 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:40:00.500497 9892 tablet_bootstrap.cc:492] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Bootstrap starting.
I20260808 10:40:00.501889 9486 heartbeater.cc:503] Master 127.1.194.125:40471 was elected leader, sending a full tablet report...
I20260808 10:40:00.502542 9892 tablet_bootstrap.cc:654] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:00.503355 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.499200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:42653)
0808 10:40:00.503162 (+ 3962us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.503230 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3873,"thread_start_us":96,"threads_started":1}
I20260808 10:40:00.503664 9639 heartbeater.cc:503] Master 127.1.194.125:40471 was elected leader, sending a full tablet report...
I20260808 10:40:00.503844 9892 tablet_bootstrap.cc:492] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: No bootstrap required, opened a new log
I20260808 10:40:00.503947 9892 ts_tablet_manager.cc:1397] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:40:00.504531 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.504265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:38831)
0808 10:40:00.504419 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.504468 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 10:40:00.504654 9892 raft_consensus.cc:348] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.504781 9892 raft_consensus.cc:374] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:00.504835 9892 raft_consensus.cc:729] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb41e7dd76845ba8af80e6db0b66d68, State: Initialized, Role: FOLLOWER
I20260808 10:40:00.504999 9892 consensus_queue.cc:260] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.505390 9892 ts_tablet_manager.cc:1428] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:40:00.506511 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.506077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.67:58989)
0808 10:40:00.506355 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.506416 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":107,"threads_started":1}
I20260808 10:40:00.660167 9894 raft_consensus.cc:482] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:00.660336 9894 raft_consensus.cc:504] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.661521 9894 leader_election.cc:288] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981), 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:40:00.665921 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.661908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:44861 (local address 127.1.194.67:39981)
0808 10:40:00.662193 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.662197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.662704 (+ 507us) server_negotiation.cc:407] Connection header received
0808 10:40:00.662766 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.662769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.662832 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.662914 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.663202 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.663795 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.664878 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.665262 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.665408 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.665460 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.665715 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.665795 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:40:00.665921 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.661863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:39121 (local address 127.1.194.65:47533)
0808 10:40:00.662024 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.662043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.662180 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.662580 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.662583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.662611 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.662764 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.662771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.663708 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.663712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.664711 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.664720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.665406 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.665410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.665606 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.665624 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.665724 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.665794 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":70}
I20260808 10:40:00.665934 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.661978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:47533 (local address 127.1.194.66:39121)
0808 10:40:00.662146 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.662151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.662181 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:40:00.662347 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.662349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.662399 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.662528 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:00.662923 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.663582 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.664835 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.665254 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.665388 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.665445 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.665752 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.665834 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:40:00.665930 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.661864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:39981 (local address 127.1.194.65:44861)
0808 10:40:00.662591 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.662606 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:00.662701 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:00.662947 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:00.662949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:00.662967 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:00.663078 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.663084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.663913 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.663916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.664715 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:00.664724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.665414 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.665418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:00.665577 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.665602 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:40:00.665665 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:40:00.665753 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":336,"thread_start_us":191,"threads_started":1}
I20260808 10:40:00.666649 9583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49df5aaae4a4a8aae429cca4eca5e6f" candidate_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f336cdb47444bc92539988dac37388" is_pre_election: true
I20260808 10:40:00.666826 9583 raft_consensus.cc:2454] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ffd8c5865d476eb0c12d3ebc1895da in term 0.
I20260808 10:40:00.666769 9732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49df5aaae4a4a8aae429cca4eca5e6f" candidate_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" is_pre_election: true
I20260808 10:40:00.667033 9732 raft_consensus.cc:2454] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ffd8c5865d476eb0c12d3ebc1895da in term 0.
I20260808 10:40:00.667270 9364 leader_election.cc:302] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82f336cdb47444bc92539988dac37388, 90ffd8c5865d476eb0c12d3ebc1895da; no voters:
I20260808 10:40:00.667552 9894 raft_consensus.cc:2781] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:00.667627 9894 raft_consensus.cc:482] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:00.667657 9894 raft_consensus.cc:3037] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:00.668841 9894 raft_consensus.cc:504] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.669287 9894 leader_election.cc:288] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [CANDIDATE]: Term 1 election: Requested vote from peers 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981), 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121)
I20260808 10:40:00.669710 9732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49df5aaae4a4a8aae429cca4eca5e6f" candidate_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb41e7dd76845ba8af80e6db0b66d68"
I20260808 10:40:00.669764 9583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49df5aaae4a4a8aae429cca4eca5e6f" candidate_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f336cdb47444bc92539988dac37388"
I20260808 10:40:00.669860 9732 raft_consensus.cc:3037] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:00.669893 9583 raft_consensus.cc:3037] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:00.670845 9583 raft_consensus.cc:2454] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ffd8c5865d476eb0c12d3ebc1895da in term 1.
I20260808 10:40:00.670846 9732 raft_consensus.cc:2454] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ffd8c5865d476eb0c12d3ebc1895da in term 1.
I20260808 10:40:00.671170 9362 leader_election.cc:302] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb41e7dd76845ba8af80e6db0b66d68, 90ffd8c5865d476eb0c12d3ebc1895da; no voters:
I20260808 10:40:00.671382 9894 raft_consensus.cc:2781] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:00.671676 9894 raft_consensus.cc:686] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [term 1 LEADER]: Becoming Leader. State: Replica: 90ffd8c5865d476eb0c12d3ebc1895da, State: Running, Role: LEADER
I20260808 10:40:00.671945 9894 consensus_queue.cc:237] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } }
I20260808 10:40:00.672446 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.672221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:54821)
0808 10:40:00.672321 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.672381 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 10:40:00.674823 9092 catalog_manager.cc:5495] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da reported cstate change: term changed from 0 to 1, leader changed from <none> to 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65). New cstate: current_term: 1 leader_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ffd8c5865d476eb0c12d3ebc1895da" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 } health_report { overall_health: UNKNOWN } } }
I20260808 10:40:00.730741 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.726764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.1.194.65:40449)
0808 10:40:00.726958 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:00.726962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:00.726975 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:40:00.727057 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:00.727061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:00.727114 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:00.727204 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:00.727520 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.728043 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.728938 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:00.729276 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:00.729375 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:00.729688 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:00.730200 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:00.730347 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:00.730483 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 10:40:00.730544 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 10:40:00.734064 9583 raft_consensus.cc:1261] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Refusing update from remote peer 90ffd8c5865d476eb0c12d3ebc1895da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:00.734392 9732 raft_consensus.cc:1261] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Refusing update from remote peer 90ffd8c5865d476eb0c12d3ebc1895da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:00.734664 9894 consensus_queue.cc:1035] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [LEADER]: Connected to new peer: Peer: permanent_uuid: "82f336cdb47444bc92539988dac37388" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:00.735010 9901 consensus_queue.cc:1035] T e49df5aaae4a4a8aae429cca4eca5e6f P 90ffd8c5865d476eb0c12d3ebc1895da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb41e7dd76845ba8af80e6db0b66d68" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:00.737098 9781 heartbeater.cc:503] Master 127.1.194.125:40471 was elected leader, sending a full tablet report...
I20260808 10:40:00.738035 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.737609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.67:34589)
0808 10:40:00.737853 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.737919 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260808 10:40:00.738863 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.738599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:56167)
0808 10:40:00.738739 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.738795 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 10:40:00.740754 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.738574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:51293)
0808 10:40:00.740577 (+ 2003us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.740681 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1829}
I20260808 10:40:00.741858 9906 mvcc.cc:205] Tried to move back new op lower bound from 7316216220601102336 to 7316216220355866624. Current Snapshot: MvccSnapshot[applied={T|T < 7316216220601102336}]
I20260808 10:40:00.742012 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.741645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.65:55277)
0808 10:40:00.741784 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.741832 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 10:40:00.742093 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.741229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.66:46567)
0808 10:40:00.741888 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.741942 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":114,"threads_started":1}
I20260808 10:40:00.743137 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.742565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42389 (local address 127.1.194.67:42899)
0808 10:40:00.743014 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.743068 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360}
I20260808 10:40:00.744022 9909 mvcc.cc:205] Tried to move back new op lower bound from 7316216220601102336 to 7316216220355866624. Current Snapshot: MvccSnapshot[applied={T|T < 7316216220601102336}]
I20260808 10:40:00.749850 9093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:00.750044 9093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:00.755067 9093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd7313f308b646f1af7676abc36a516e: Sending DeleteTablet for 3 replicas of tablet e49df5aaae4a4a8aae429cca4eca5e6f
I20260808 10:40:00.755988 9561 tablet_service.cc:1514] Processing DeleteTablet for tablet e49df5aaae4a4a8aae429cca4eca5e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:00 UTC) from {username='oryx', principal='oryx/127.1.194.125@KRBTEST.COM'} at 127.0.0.1:38966
I20260808 10:40:00.756199 9416 tablet_service.cc:1514] Processing DeleteTablet for tablet e49df5aaae4a4a8aae429cca4eca5e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:00 UTC) from {username='oryx', principal='oryx/127.1.194.125@KRBTEST.COM'} at 127.0.0.1:55612
I20260808 10:40:00.756460 9918 tablet_replica.cc:330] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: stopping tablet replica
I20260808 10:40:00.756533 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 9248
I20260808 10:40:00.756578 9711 tablet_service.cc:1514] Processing DeleteTablet for tablet e49df5aaae4a4a8aae429cca4eca5e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:00 UTC) from {username='oryx', principal='oryx/127.1.194.125@KRBTEST.COM'} at 127.0.0.1:46588
I20260808 10:40:00.756614 9918 raft_consensus.cc:2229] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:00.756786 9918 raft_consensus.cc:2258] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:00.758562 9918 ts_tablet_manager.cc:1905] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:00.759315 9920 tablet_replica.cc:330] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: stopping tablet replica
I20260808 10:40:00.759477 9920 raft_consensus.cc:2229] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:00.759598 9920 raft_consensus.cc:2258] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:00.760448 9920 ts_tablet_manager.cc:1905] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:00.761922 9918 ts_tablet_manager.cc:1918] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:00.762038 9918 log.cc:1196] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/e49df5aaae4a4a8aae429cca4eca5e6f
I20260808 10:40:00.762384 9918 ts_tablet_manager.cc:1939] T e49df5aaae4a4a8aae429cca4eca5e6f P 82f336cdb47444bc92539988dac37388: Deleting consensus metadata
I20260808 10:40:00.763232 9078 catalog_manager.cc:4841] TS 82f336cdb47444bc92539988dac37388 (127.1.194.66:39121): tablet e49df5aaae4a4a8aae429cca4eca5e6f (table test-table [id=496a0b0c1aa849b6af6d556a1a494f9b]) successfully deleted
I20260808 10:40:00.763772 9920 ts_tablet_manager.cc:1918] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:00.763865 9920 log.cc:1196] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/e49df5aaae4a4a8aae429cca4eca5e6f
I20260808 10:40:00.764184 9920 ts_tablet_manager.cc:1939] T e49df5aaae4a4a8aae429cca4eca5e6f P 2cb41e7dd76845ba8af80e6db0b66d68: Deleting consensus metadata
I20260808 10:40:00.764990 9075 catalog_manager.cc:4841] TS 2cb41e7dd76845ba8af80e6db0b66d68 (127.1.194.67:39981): tablet e49df5aaae4a4a8aae429cca4eca5e6f (table test-table [id=496a0b0c1aa849b6af6d556a1a494f9b]) successfully deleted
I20260808 10:40:00.766516 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 9493
I20260808 10:40:00.766638 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:00.766323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40449 (local address 127.0.0.1:55630)
0808 10:40:00.766465 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:00.766525 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:40449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 10:40:00.766837 9075 catalog_manager.cc:4584] TS 90ffd8c5865d476eb0c12d3ebc1895da (127.1.194.65:40449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e49df5aaae4a4a8aae429cca4eca5e6f: Network error: Client connection negotiation failed: client connection to 127.1.194.65:40449: connect: Connection refused (error 111)
I20260808 10:40:00.775008 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 9644
I20260808 10:40:00.784041 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 8942
I20260808 10:40:00.791396 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 9062
2026-08-08T10:40:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5930 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:40:00 dist-test-slave-ss4n krb5kdc[9924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:40:00 dist-test-slave-ss4n krb5kdc[9924](info): set up 2 sockets
Aug 08 10:40:00 dist-test-slave-ss4n krb5kdc[9924](info): commencing operation
krb5kdc: starting...
W20260808 10:40:01.881960 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:40:01 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab.
Entry for principal oryx/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab.
I20260808 10:40:02.025372 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46465
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab
--principal=oryx/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:46465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:02.139768 9937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:02.140126 9937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:02.140209 9937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:02.144335 9937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:02.144438 9937 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:40:02.144474 9937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:02.144516 9937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:02.144558 9937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:02.144595 9937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:02.150060 9937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:46465
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.126.keytab
--principal=oryx/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:02.151206 9937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:02.152249 9937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:02.160928 9944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:02.161079 9937 server_base.cc:1034] running on GCE node
W20260808 10:40:02.160984 9947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.161325 9943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:02.161973 9937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:02.163189 9937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:02.164366 9937 hybrid_clock.cc:648] HybridClock initialized: now 1786185602164334 us; error 36 us; skew 500 ppm
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:02.173254 9937 init.cc:377] Logged in from keytab as oryx/127.1.194.126@KRBTEST.COM (short username oryx)
I20260808 10:40:02.174362 9937 webserver.cc:460] Webserver started at http://127.1.194.126:39259/ using document root <none> and password file <none>
I20260808 10:40:02.174925 9937 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:02.174989 9937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:02.175163 9937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:02.177233 9937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "dc4037197b0a4b409009d5b511cae8ca"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "47b1df9a1c01e60b153c799d83016022"
server_key_iv: "76a40a70409842aae454dfbe80c93bda"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.177834 9937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "dc4037197b0a4b409009d5b511cae8ca"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "47b1df9a1c01e60b153c799d83016022"
server_key_iv: "76a40a70409842aae454dfbe80c93bda"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.181538 9937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:02.183998 9954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.185294 9937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:02.185457 9937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "dc4037197b0a4b409009d5b511cae8ca"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "47b1df9a1c01e60b153c799d83016022"
server_key_iv: "76a40a70409842aae454dfbe80c93bda"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.185652 9937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:02.197395 9937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:02.200997 9937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:02.201232 9937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:02.209887 9937 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:46465
I20260808 10:40:02.209904 10022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:46465 every 8 connection(s)
I20260808 10:40:02.210871 9937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:40:02.213414 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 9937
I20260808 10:40:02.213582 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:40:02.213753 10024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:02.213899 1801 external_mini_cluster.cc:1428] Setting key 6d9bf5b0362bcc213f1653b7a92b4a08
I20260808 10:40:02.220976 10024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Bootstrap starting.
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:02.223999 10024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:02.224931 10024 log.cc:824] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:02.227519 10024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: No bootstrap required, opened a new log
I20260808 10:40:02.230430 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.215380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.1.194.126:46465)
0808 10:40:02.215807 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:02.215822 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:02.215855 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:40:02.216511 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:02.216530 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:02.216849 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:02.217131 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:02.218129 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.219165 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.220367 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.220839 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.220958 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.224314 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:02.224321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:02.224338 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:02.224373 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:02.227151 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.227794 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.227800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.227813 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.227970 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.228311 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.228315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.228318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.228760 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:02.229011 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:02.229323 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 10:40:02.229691 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":139,"threads_started":1}
I20260808 10:40:02.230764 10024 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4037197b0a4b409009d5b511cae8ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46465 } }
I20260808 10:40:02.231071 10024 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:02.231165 10024 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc4037197b0a4b409009d5b511cae8ca, State: Initialized, Role: FOLLOWER
I20260808 10:40:02.231607 10024 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4037197b0a4b409009d5b511cae8ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46465 } }
I20260808 10:40:02.231768 10024 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:40:02.231822 10024 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:40:02.231921 10024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:02.233156 10024 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4037197b0a4b409009d5b511cae8ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46465 } }
I20260808 10:40:02.233443 10024 leader_election.cc:302] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc4037197b0a4b409009d5b511cae8ca; no voters:
I20260808 10:40:02.233781 10024 leader_election.cc:288] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:40:02.233955 10029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:02.234221 10029 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [term 1 LEADER]: Becoming Leader. State: Replica: dc4037197b0a4b409009d5b511cae8ca, State: Running, Role: LEADER
I20260808 10:40:02.234524 10029 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4037197b0a4b409009d5b511cae8ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46465 } }
I20260808 10:40:02.235052 10024 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:40:02.236244 10030 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc4037197b0a4b409009d5b511cae8ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4037197b0a4b409009d5b511cae8ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46465 } } }
I20260808 10:40:02.236244 10031 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc4037197b0a4b409009d5b511cae8ca. Latest consensus state: current_term: 1 leader_uuid: "dc4037197b0a4b409009d5b511cae8ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4037197b0a4b409009d5b511cae8ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46465 } } }
I20260808 10:40:02.236397 10030 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:02.236478 10031 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:02.236822 10038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:02.240016 10038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:02.245210 10038 catalog_manager.cc:1347] Generated new cluster ID: de965a27c440423887c2430056600175
I20260808 10:40:02.245283 10038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:02.257864 10038 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:40:02.259203 10038 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:02.273273 10038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Generated new TSK 0
I20260808 10:40:02.274112 10038 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab.
Entry for principal oryx/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab.
I20260808 10:40:02.317345 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:46465
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:02.429865 10053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:02.430145 10053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:02.430235 10053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:02.434381 10053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:02.434443 10053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:02.434484 10053 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:40:02.434561 10053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:40:02.440058 10053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:46465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:02.441190 10053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:02.442276 10053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:02.452045 10060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.452066 10059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.452071 10062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:02.452109 10053 server_base.cc:1034] running on GCE node
I20260808 10:40:02.452708 10053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:02.453346 10053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:02.454542 10053 hybrid_clock.cc:648] HybridClock initialized: now 1786185602454519 us; error 32 us; skew 500 ppm
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:02.463128 10053 init.cc:377] Logged in from keytab as oryx/127.1.194.65@KRBTEST.COM (short username oryx)
I20260808 10:40:02.464128 10053 webserver.cc:460] Webserver started at http://127.1.194.65:38599/ using document root <none> and password file <none>
I20260808 10:40:02.464663 10053 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:02.464720 10053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:02.464885 10053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:02.467000 10053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "b24a71873a1543ccad0413b817c7a22b"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "4a4ab27dbe607a386292a103d7ff5a9c"
server_key_iv: "eeb5e81fa0b43110ac2bc41e28dc504f"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.467519 10053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "b24a71873a1543ccad0413b817c7a22b"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "4a4ab27dbe607a386292a103d7ff5a9c"
server_key_iv: "eeb5e81fa0b43110ac2bc41e28dc504f"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.471218 10053 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:40:02.473708 10069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.475035 10053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:02.475139 10053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "b24a71873a1543ccad0413b817c7a22b"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "4a4ab27dbe607a386292a103d7ff5a9c"
server_key_iv: "eeb5e81fa0b43110ac2bc41e28dc504f"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.475235 10053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:02.499759 10053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:02.502810 10053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:02.502969 10053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:02.504370 10053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:02.504438 10053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.504505 10053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:02.504547 10053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:02.515971 10053 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:44647
I20260808 10:40:02.516355 10193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:44647 every 8 connection(s)
I20260808 10:40:02.516970 10053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:40:02.519222 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.505324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:52627 (local address 127.1.194.126:46465)
0808 10:40:02.505450 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:02.505454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:02.506537 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 10:40:02.507685 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:02.507691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:02.507758 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:02.507836 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:02.509635 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.510564 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.512034 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.512462 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.512580 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.515328 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:02.515334 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:02.515337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:02.515368 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:02.517141 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.517789 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.517792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.517794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.517902 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.518263 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.518266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.518268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.518455 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:02.518533 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:02.518995 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 10:40:02.519102 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:40:02.519994 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.505890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.1.194.65:52627)
0808 10:40:02.506387 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:02.506429 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:02.507392 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:02.508299 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:02.508307 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:02.508760 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:02.509429 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.509452 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.510723 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.510727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.511902 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.511911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.512675 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.512681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.512844 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.513464 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:02.513526 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:02.515153 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:02.517256 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.517262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.517276 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.517669 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.518002 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.518004 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.518005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.518155 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.518564 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:02.518568 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:02.518785 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 10:40:02.518930 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 10:40:02.519187 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":141,"threads_started":1}
I20260808 10:40:02.521251 10195 heartbeater.cc:347] Connected to a master server at 127.1.194.126:46465
I20260808 10:40:02.521387 10195 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:02.521951 10195 heartbeater.cc:511] Master 127.1.194.126:46465 requested a full tablet report, sending...
I20260808 10:40:02.523617 9972 ts_manager.cc:194] Registered new tserver with Master: b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647)
I20260808 10:40:02.524896 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 10053
I20260808 10:40:02.524960 9972 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:52627
I20260808 10:40:02.525048 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:40:02.525367 1801 external_mini_cluster.cc:1428] Setting key 60609857944a501248b88b29fdd570b6
I20260808 10:40:02.532385 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.523684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:41481 (local address 127.1.194.126:46465)
0808 10:40:02.523893 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:02.523897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:02.523980 (+ 83us) server_negotiation.cc:407] Connection header received
0808 10:40:02.524118 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:02.524125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:02.524164 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:02.524250 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:02.524708 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.525338 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.526466 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.526730 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.526834 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.528397 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:02.528400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:02.528402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:02.528425 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:02.530496 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.531129 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.531132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.531133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.531221 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.531532 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.531582 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.531583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.531765 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:02.531845 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:02.532158 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 10:40:02.532261 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260808 10:40:02.532620 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.523602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.1.194.65:41481)
0808 10:40:02.523910 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:02.523926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:02.524023 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:02.524265 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:02.524268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:02.524477 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:02.524579 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.524584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.525559 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.525562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.526347 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.526356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.526970 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.526973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.527072 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.527711 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:02.527730 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:02.528246 (+ 516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:02.530686 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.530689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.530690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.531018 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.531323 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.531329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.531334 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.531440 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.531864 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:02.531870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:02.531990 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 10:40:02.532106 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:40:02.532294 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for oryx/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab.
Entry for principal oryx/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab.
I20260808 10:40:02.556947 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:46465
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:02.689078 10204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:02.689422 10204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:02.689473 10204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:02.693769 10204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:02.693861 10204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:02.693928 10204 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:40:02.693994 10204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:40:02.699388 10204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:46465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:02.700481 10204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:02.701520 10204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:02.711211 10209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.711256 10210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.711236 10212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:02.711685 10204 server_base.cc:1034] running on GCE node
I20260808 10:40:02.712100 10204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:02.712754 10204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:02.713938 10204 hybrid_clock.cc:648] HybridClock initialized: now 1786185602713910 us; error 46 us; skew 500 ppm
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:02.717043 10204 init.cc:377] Logged in from keytab as oryx/127.1.194.66@KRBTEST.COM (short username oryx)
I20260808 10:40:02.718211 10204 webserver.cc:460] Webserver started at http://127.1.194.66:36229/ using document root <none> and password file <none>
I20260808 10:40:02.718827 10204 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:02.718921 10204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:02.719148 10204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:02.721268 10204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "f6cd724bd644498396b3f12b7c9cc054"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "d8fffdcb00e59c07d7a6e565f234690d"
server_key_iv: "19c52b7bccf78949afe8723945105a9c"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.721915 10204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "f6cd724bd644498396b3f12b7c9cc054"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "d8fffdcb00e59c07d7a6e565f234690d"
server_key_iv: "19c52b7bccf78949afe8723945105a9c"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.726024 10204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:40:02.728705 10219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.730264 10204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:02.730406 10204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "f6cd724bd644498396b3f12b7c9cc054"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "d8fffdcb00e59c07d7a6e565f234690d"
server_key_iv: "19c52b7bccf78949afe8723945105a9c"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.730543 10204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:02.756685 10204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:02.760269 10204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:02.760499 10204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:02.762239 10204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:02.762317 10204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.762403 10204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:02.762460 10204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:02.774966 10204 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:41065
I20260808 10:40:02.775053 10343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:41065 every 8 connection(s)
I20260808 10:40:02.776157 10204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:40:02.778728 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.763187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:44701 (local address 127.1.194.126:46465)
0808 10:40:02.763363 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:02.763370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:02.764338 (+ 968us) server_negotiation.cc:407] Connection header received
0808 10:40:02.765620 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:02.765624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:02.765695 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:02.765806 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:02.767236 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.767895 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.769310 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.769691 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.769834 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.774101 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:02.774108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:02.774118 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:02.774149 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:02.776250 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.776993 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.776997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.776999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.777132 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.777592 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.777595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.777597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.777810 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:02.777908 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:02.778486 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 10:40:02.778608 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:40:02.779770 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.763636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.1.194.66:44701)
0808 10:40:02.764145 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:02.764203 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:02.765340 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:02.766029 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:02.766041 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:02.766502 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:02.767043 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.767065 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.768070 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.768074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.769160 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.769170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.769885 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.769890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.770094 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.770870 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:02.770895 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:02.773893 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:02.776417 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.776422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.776433 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.776839 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.777242 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.777246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.777248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.777398 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.777913 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:02.777918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:02.778198 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 10:40:02.778428 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 10:40:02.778782 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":139,"threads_started":1}
I20260808 10:40:02.780964 10344 heartbeater.cc:347] Connected to a master server at 127.1.194.126:46465
I20260808 10:40:02.781152 10344 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:02.781689 10344 heartbeater.cc:511] Master 127.1.194.126:46465 requested a full tablet report, sending...
I20260808 10:40:02.782817 9973 ts_manager.cc:194] Registered new tserver with Master: f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065)
I20260808 10:40:02.783663 9973 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:44701
I20260808 10:40:02.784788 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 10204
I20260808 10:40:02.784917 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:40:02.785231 1801 external_mini_cluster.cc:1428] Setting key f2d5d7e12acfb62dfd8ccf4fd81e4327
I20260808 10:40:02.791313 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.783301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.1.194.66:52895)
0808 10:40:02.783639 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:02.783652 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:02.783732 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:02.784005 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:02.784008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:02.784197 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:02.784336 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.784343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.785542 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.785554 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.786166 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.786175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.786629 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.786632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.786727 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.787265 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:02.787276 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:02.787583 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:02.789783 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.789787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.789789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.790021 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.790316 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:02.790319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:02.790320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:02.790397 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:02.790905 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:02.790907 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:02.790985 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 10:40:02.791071 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:40:02.791180 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":98,"threads_started":1}
I20260808 10:40:02.791508 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.783376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:52895 (local address 127.1.194.126:46465)
0808 10:40:02.783506 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:02.783511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:02.783725 (+ 214us) server_negotiation.cc:407] Connection header received
0808 10:40:02.783826 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:02.783830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:02.783872 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:02.784007 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:02.784560 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.785383 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.786289 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.786517 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.786628 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.787717 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:02.787721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:02.787724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:02.787747 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:02.789577 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.790128 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.790135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.790139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.790231 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:02.790485 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:02.790491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:02.790496 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:02.790800 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:02.790909 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:02.791155 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 10:40:02.791302 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.1.194.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab.
Entry for principal oryx/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab.
I20260808 10:40:02.816509 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:46465
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:02.930814 10353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:02.931064 10353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:02.931108 10353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:02.935222 10353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:02.935284 10353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:02.935324 10353 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:40:02.935356 10353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:40:02.940886 10353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.194.126:46465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:02.941924 10353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:02.942935 10353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:02.952455 10358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.952498 10359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:02.952459 10361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:02.952739 10353 server_base.cc:1034] running on GCE node
I20260808 10:40:02.953366 10353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:02.954167 10353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:02.955358 10353 hybrid_clock.cc:648] HybridClock initialized: now 1786185602955341 us; error 26 us; skew 500 ppm
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:02.958235 10353 init.cc:377] Logged in from keytab as oryx/127.1.194.67@KRBTEST.COM (short username oryx)
I20260808 10:40:02.959200 10353 webserver.cc:460] Webserver started at http://127.1.194.67:45257/ using document root <none> and password file <none>
I20260808 10:40:02.959798 10353 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:02.959897 10353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:02.960151 10353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:02.962407 10353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "5a3cce12a2044c2a95327b261568d029"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "5f5108a37a91914d226df9a2932afc01"
server_key_iv: "69818263bff5e4442fdc51ab659c9bb5"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.963004 10353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "5a3cce12a2044c2a95327b261568d029"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "5f5108a37a91914d226df9a2932afc01"
server_key_iv: "69818263bff5e4442fdc51ab659c9bb5"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.966964 10353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:40:02.969386 10368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.970765 10353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:40:02.970984 10353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "5a3cce12a2044c2a95327b261568d029"
format_stamp: "Formatted at 2026-08-08 10:40:02 on dist-test-slave-ss4n"
server_key: "5f5108a37a91914d226df9a2932afc01"
server_key_iv: "69818263bff5e4442fdc51ab659c9bb5"
server_key_version: "encryptionkey@0"
I20260808 10:40:02.971122 10353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:02.983309 10353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:02.986841 10353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:02.987066 10353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:02.988546 10353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:02.988623 10353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:02.988703 10353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:02.988749 10353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:40:02 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:03.000880 10353 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:36651
I20260808 10:40:03.000931 10495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:36651 every 8 connection(s)
I20260808 10:40:03.002060 10353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:40:03.003811 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 10353
I20260808 10:40:03.003960 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:40:03.004244 1801 external_mini_cluster.cc:1428] Setting key 757b228950bbbb670847d388b900d62b
I20260808 10:40:03.004771 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.989393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:56111 (local address 127.1.194.126:46465)
0808 10:40:02.989578 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:02.989583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:02.990571 (+ 988us) server_negotiation.cc:407] Connection header received
0808 10:40:02.991994 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:02.992001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:02.992078 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:02.992153 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:02.994204 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.994870 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.996929 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.997301 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.997416 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.000571 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:03.000576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:03.000580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:03.000612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:03.002348 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.003015 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.003017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.003019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.003113 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.003496 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.003499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.003501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.003769 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:03.003882 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.004477 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.004608 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:40:03.005762 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:02.989863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.1.194.67:56111)
0808 10:40:02.990371 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:02.990432 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:02.991703 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:02.992664 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:02.992673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:02.993200 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:02.994025 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.994049 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.995022 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.995037 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.996788 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:02.996801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:02.997509 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:02.997515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:02.997732 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:02.998375 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:02.998403 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:03.000381 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:03.002468 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:03.002473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:03.002487 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:03.002845 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:03.003258 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:03.003261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:03.003262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:03.003395 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:03.003945 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:03.003953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:03.004213 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.004410 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.004779 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
I20260808 10:40:03.007169 10496 heartbeater.cc:347] Connected to a master server at 127.1.194.126:46465
I20260808 10:40:03.007318 10496 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:03.007794 10496 heartbeater.cc:511] Master 127.1.194.126:46465 requested a full tablet report, sending...
I20260808 10:40:03.008788 9973 ts_manager.cc:194] Registered new tserver with Master: 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.009577 9973 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:56111
I20260808 10:40:03.016327 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.009307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:45351 (local address 127.1.194.126:46465)
0808 10:40:03.009404 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.009408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.009706 (+ 298us) server_negotiation.cc:407] Connection header received
0808 10:40:03.009830 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.009833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.009874 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.009958 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:03.010410 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.010902 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.011721 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.011932 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.012047 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.013122 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:03.013126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:03.013128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:03.013158 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:03.014726 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.015157 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.015161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.015163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.015243 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.015519 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.015523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.015528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.015698 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:03.015808 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.016070 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.016177 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:40:03.016337 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.009232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.1.194.67:45351)
0808 10:40:03.009598 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.009612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.009725 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.010005 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.010007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.010189 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:03.010287 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.010292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.011029 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.011032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.011595 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.011603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.012043 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.012047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.012137 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.012663 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:03.012678 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:03.012985 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:03.014856 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:03.014860 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:03.014862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:03.015058 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:03.015336 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:03.015340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:03.015342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:03.015428 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:03.015797 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:03.015800 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:03.015897 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.016004 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.016141 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":116,"threads_started":1}
I20260808 10:40:03.018177 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185603, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:40:03.043541 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:03.037085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.1.194.126:46465)
0808 10:40:03.037279 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.037283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.037295 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:40:03.037337 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.037340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.037379 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.037474 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:03.038253 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.038819 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.039708 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.039929 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.040051 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.043419 (+ 3368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51014: BlockingRecv error: recv got EOF from 127.0.0.1:51014 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185603, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:03.054195 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.045360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.1.194.126:46465)
0808 10:40:03.045601 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.045606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.045683 (+ 77us) server_negotiation.cc:407] Connection header received
0808 10:40:03.045850 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.045853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.045902 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.045985 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:03.046600 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.047107 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.047904 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.048125 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.048235 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.051125 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:03.051128 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:03.051130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:03.051154 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:03.052550 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.053004 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.053007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.053008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.053078 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.053367 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.053369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.053370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.053641 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:03.053750 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.053987 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.054078 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:40:03.057356 9973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:40:03.060446 9973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:03.074743 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.069335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.1.194.67:36651)
0808 10:40:03.069779 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.069790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.069811 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:40:03.069884 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.069889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.070087 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.070635 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.070764 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.071914 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.073370 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.074064 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.074187 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.074351 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.074490 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.074579 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":165,"threads_started":1}
I20260808 10:40:03.074733 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.068887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36651 (local address 127.0.0.1:39542)
0808 10:40:03.069333 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.069346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.069458 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.070268 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.070274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.070309 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.070464 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.070473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.072044 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.072047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.073232 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.073245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.074198 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.074201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.074304 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.074346 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.074451 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.074579 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":9,"spinlock_wait_cycles":12800,"thread_start_us":128,"threads_started":1}
I20260808 10:40:03.074956 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.068691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44647 (local address 127.0.0.1:53650)
0808 10:40:03.069258 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.069308 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.069458 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.070268 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.070274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.070308 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.070639 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.070649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.071760 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.071766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.073269 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.073278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.074342 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.074346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.074437 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.074467 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.074534 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.074611 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":184,"spinlock_wait_cycles":36096,"thread_start_us":70,"threads_started":1}
I20260808 10:40:03.075361 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.069382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:41065 (local address 127.0.0.1:49256)
0808 10:40:03.070449 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.070463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.070549 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.071033 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.071035 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.071055 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.071138 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.071143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.072362 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.072368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.073581 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.073592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.074793 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.074796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.075078 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.075107 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.075176 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.075244 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":45,"thread_start_us":139,"threads_started":1}
I20260808 10:40:03.075755 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.069842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.1.194.66:41065)
0808 10:40:03.070287 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.070300 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.070541 (+ 241us) server_negotiation.cc:407] Connection header received
0808 10:40:03.070678 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.070683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.070860 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.071033 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.071255 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.072254 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.073711 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.074484 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.074582 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.074730 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.075532 (+ 802us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.075622 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":124,"threads_started":1}
I20260808 10:40:03.076249 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.068856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.1.194.65:44647)
0808 10:40:03.069779 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.069788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.069811 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:40:03.069884 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.069889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.070095 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.070384 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.070774 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.071649 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.073409 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.074194 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.074822 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.075985 (+ 1163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.076034 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.076124 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":70,"threads_started":1}
I20260808 10:40:03.077980 10420 tablet_service.cc:1467] Processing CreateTablet for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 (DEFAULT_TABLE table=test-table [id=3c208e907b844593a2e3ab03c1e66465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.078177 10270 tablet_service.cc:1467] Processing CreateTablet for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 (DEFAULT_TABLE table=test-table [id=3c208e907b844593a2e3ab03c1e66465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.078925 10420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.078663 10121 tablet_service.cc:1467] Processing CreateTablet for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 (DEFAULT_TABLE table=test-table [id=3c208e907b844593a2e3ab03c1e66465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.079151 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.079542 10121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.085697 10527 tablet_bootstrap.cc:492] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Bootstrap starting.
I20260808 10:40:03.087746 10527 tablet_bootstrap.cc:654] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.087785 10528 tablet_bootstrap.cc:492] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Bootstrap starting.
I20260808 10:40:03.088460 10527 log.cc:824] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:03.089068 10530 tablet_bootstrap.cc:492] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Bootstrap starting.
I20260808 10:40:03.089787 10528 tablet_bootstrap.cc:654] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.090590 10528 log.cc:824] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:03.091126 10527 tablet_bootstrap.cc:492] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: No bootstrap required, opened a new log
I20260808 10:40:03.091148 10530 tablet_bootstrap.cc:654] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.091331 10527 ts_tablet_manager.cc:1397] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:40:03.091919 10530 log.cc:824] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:03.093736 10528 tablet_bootstrap.cc:492] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: No bootstrap required, opened a new log
I20260808 10:40:03.093914 10530 tablet_bootstrap.cc:492] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: No bootstrap required, opened a new log
I20260808 10:40:03.093936 10528 ts_tablet_manager.cc:1397] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:40:03.094091 10530 ts_tablet_manager.cc:1397] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:40:03.095427 10527 raft_consensus.cc:348] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.095700 10527 raft_consensus.cc:374] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.095762 10527 raft_consensus.cc:729] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3cce12a2044c2a95327b261568d029, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.096227 10527 consensus_queue.cc:260] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.097246 10527 ts_tablet_manager.cc:1428] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:40:03.097651 10496 heartbeater.cc:503] Master 127.1.194.126:46465 was elected leader, sending a full tablet report...
I20260808 10:40:03.097908 10530 raft_consensus.cc:348] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.098130 10530 raft_consensus.cc:374] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.097949 10528 raft_consensus.cc:348] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.098192 10530 raft_consensus.cc:729] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.098202 10528 raft_consensus.cc:374] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.098256 10528 raft_consensus.cc:729] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.098661 10528 consensus_queue.cc:260] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.098667 10530 consensus_queue.cc:260] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.099232 10195 heartbeater.cc:503] Master 127.1.194.126:46465 was elected leader, sending a full tablet report...
I20260808 10:40:03.099581 10528 ts_tablet_manager.cc:1428] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:40:03.099592 10530 ts_tablet_manager.cc:1428] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:40:03.100229 10344 heartbeater.cc:503] Master 127.1.194.126:46465 was elected leader, sending a full tablet report...
I20260808 10:40:03.123937 10537 raft_consensus.cc:482] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:03.124168 10537 raft_consensus.cc:504] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.124926 10536 raft_consensus.cc:482] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:03.125111 10536 raft_consensus.cc:504] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.125505 10537 leader_election.cc:288] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.126268 10536 leader_election.cc:288] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651), f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065)
I20260808 10:40:03.130817 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.126251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:40125 (local address 127.1.194.65:44647)
0808 10:40:03.126388 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.126393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.126408 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:03.126480 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.126482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.126530 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.126616 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.127206 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.128006 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.129555 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.130156 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.130260 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.130307 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.130618 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.130694 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":56}
I20260808 10:40:03.130810 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.125722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44647 (local address 127.1.194.66:40125)
0808 10:40:03.125852 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.125869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.125959 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.126817 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.126823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.126845 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.127061 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.127071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.128130 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.128133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.129401 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.129409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.130374 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.130377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.130484 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.130506 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.130581 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.130665 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":31}
I20260808 10:40:03.131937 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.126271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:60843 (local address 127.1.194.67:36651)
0808 10:40:03.126478 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.126483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.126499 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:40:03.126554 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.126557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.126610 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.126700 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.127803 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.128641 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.130185 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.131332 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.131458 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.131497 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.131726 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.131813 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":716,"server-negotiator.queue_time_us":116}
I20260808 10:40:03.132045 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.126558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:49313 (local address 127.1.194.67:36651)
0808 10:40:03.126995 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.127006 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.127240 (+ 234us) server_negotiation.cc:407] Connection header received
0808 10:40:03.127440 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.127442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.127479 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.127568 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.127876 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.128789 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.130312 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.130905 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.131009 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.131056 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.131842 (+ 786us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.131897 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":365,"thread_start_us":140,"threads_started":1}
I20260808 10:40:03.131978 10142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24a71873a1543ccad0413b817c7a22b" is_pre_election: true
I20260808 10:40:03.132087 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.127180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:48619 (local address 127.1.194.66:41065)
0808 10:40:03.127284 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.127288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.127965 (+ 677us) server_negotiation.cc:407] Connection header received
0808 10:40:03.128130 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.128133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.128172 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.128214 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.128569 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.129260 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.130721 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.131201 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.131278 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.131306 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.131912 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.131987 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:40:03.132211 10142 raft_consensus.cc:2454] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 0.
I20260808 10:40:03.132330 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.125787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36651 (local address 127.1.194.66:60843)
0808 10:40:03.126098 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.126110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.126213 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.126824 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.126831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.126855 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.127672 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.127682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.128763 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.128766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.130034 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.130045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.131463 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.131467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.131589 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.131611 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.132094 (+ 483us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.132169 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":156,"thread_start_us":150,"threads_started":1}
I20260808 10:40:03.132431 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.126488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36651 (local address 127.1.194.65:49313)
0808 10:40:03.127161 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.127173 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.127308 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.127589 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.127594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.127611 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.127734 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.127762 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.128898 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.128901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.130196 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.130206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.131603 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.131606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.131720 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.131743 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.131964 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.132189 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":29,"thread_start_us":171,"threads_started":1}
I20260808 10:40:03.132745 10223 leader_election.cc:302] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.132891 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.126908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:41065 (local address 127.1.194.65:48619)
0808 10:40:03.127897 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.127910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.128019 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.128319 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.128321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.128336 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.128434 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.128440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.129393 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.129396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.130600 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.130609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.131647 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.131651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.131774 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.131797 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.132636 (+ 839us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.132709 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"thread_start_us":99,"threads_started":1}
I20260808 10:40:03.133064 10537 raft_consensus.cc:2781] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:03.133150 10537 raft_consensus.cc:482] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:03.133211 10537 raft_consensus.cc:3037] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.133347 10441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029" is_pre_election: true
I20260808 10:40:03.133404 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054" is_pre_election: true
I20260808 10:40:03.133697 10441 raft_consensus.cc:2454] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 0.
I20260808 10:40:03.133930 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029" is_pre_election: true
I20260808 10:40:03.134056 10440 raft_consensus.cc:2454] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 0.
I20260808 10:40:03.134234 10071 leader_election.cc:302] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3cce12a2044c2a95327b261568d029, b24a71873a1543ccad0413b817c7a22b; no voters:
I20260808 10:40:03.134529 10536 raft_consensus.cc:2781] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:03.134624 10536 raft_consensus.cc:482] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:03.134680 10536 raft_consensus.cc:3037] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.134707 10537 raft_consensus.cc:504] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.134943 10293 raft_consensus.cc:2379] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b24a71873a1543ccad0413b817c7a22b in current term 1: Already voted for candidate f6cd724bd644498396b3f12b7c9cc054 in this term.
I20260808 10:40:03.135440 10142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24a71873a1543ccad0413b817c7a22b"
I20260808 10:40:03.136052 10536 raft_consensus.cc:504] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.136299 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029"
I20260808 10:40:03.136400 10440 raft_consensus.cc:3037] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.136475 10536 leader_election.cc:288] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Requested vote from peers 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651), f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065)
I20260808 10:40:03.136878 10441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029"
I20260808 10:40:03.137321 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b82a9ec2d2e04845bdcd6c4e1c15ded5" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054"
I20260808 10:40:03.137473 10293 raft_consensus.cc:2379] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b24a71873a1543ccad0413b817c7a22b in current term 1: Already voted for candidate f6cd724bd644498396b3f12b7c9cc054 in this term.
I20260808 10:40:03.137701 10440 raft_consensus.cc:2454] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 1.
I20260808 10:40:03.137789 10071 leader_election.cc:302] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b; no voters: 5a3cce12a2044c2a95327b261568d029, f6cd724bd644498396b3f12b7c9cc054
I20260808 10:40:03.137964 10142 raft_consensus.cc:2379] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f6cd724bd644498396b3f12b7c9cc054 in current term 1: Already voted for candidate b24a71873a1543ccad0413b817c7a22b in this term.
I20260808 10:40:03.138031 10221 leader_election.cc:302] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3cce12a2044c2a95327b261568d029, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.138116 10536 raft_consensus.cc:2726] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 10:40:03.138410 10543 raft_consensus.cc:2781] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:03.138662 10543 raft_consensus.cc:686] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 LEADER]: Becoming Leader. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Running, Role: LEADER
I20260808 10:40:03.138679 10537 leader_election.cc:288] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 election: Requested vote from peers b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.139012 10543 consensus_queue.cc:237] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } }
I20260808 10:40:03.141889 9972 catalog_manager.cc:5495] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "f6cd724bd644498396b3f12b7c9cc054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } health_report { overall_health: HEALTHY } } }
I20260808 10:40:03.159502 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.155914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.1.194.66:41065)
0808 10:40:03.156059 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.156063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.156091 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:40:03.156160 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.156162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.156213 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.156292 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.156499 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.156996 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.157950 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.158167 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.158264 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.158673 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.159112 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.159195 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.159300 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.159348 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:40:03.165230 10440 raft_consensus.cc:1261] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Refusing update from remote peer f6cd724bd644498396b3f12b7c9cc054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.165241 10142 raft_consensus.cc:1261] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Refusing update from remote peer f6cd724bd644498396b3f12b7c9cc054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.165876 10544 consensus_queue.cc:1035] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.166019 10543 consensus_queue.cc:1035] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.172923 10547 mvcc.cc:205] Tried to move back new op lower bound from 7316216230558171136 to 7316216230459801600. Current Snapshot: MvccSnapshot[applied={T|T < 7316216230558171136}]
I20260808 10:40:03.172996 10548 mvcc.cc:205] Tried to move back new op lower bound from 7316216230558171136 to 7316216230459801600. Current Snapshot: MvccSnapshot[applied={T|T < 7316216230558171136}]
I20260808 10:40:03.182565 9972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.182768 9972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.185232 9972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Sending DeleteTablet for 3 replicas of tablet b82a9ec2d2e04845bdcd6c4e1c15ded5
I20260808 10:40:03.186107 10121 tablet_service.cc:1514] Processing DeleteTablet for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:53650
I20260808 10:40:03.186195 10420 tablet_service.cc:1514] Processing DeleteTablet for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39542
I20260808 10:40:03.186640 10270 tablet_service.cc:1514] Processing DeleteTablet for tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:49256
I20260808 10:40:03.186959 10561 tablet_replica.cc:330] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: stopping tablet replica
I20260808 10:40:03.187119 10562 tablet_replica.cc:330] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: stopping tablet replica
I20260808 10:40:03.187283 10561 raft_consensus.cc:2229] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:03.187425 10562 raft_consensus.cc:2229] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:40:03.187621 10561 raft_consensus.cc:2258] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.187868 10562 raft_consensus.cc:2258] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.188006 10560 tablet_replica.cc:330] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: stopping tablet replica
I20260808 10:40:03.188086 9972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:40:03.188273 10560 raft_consensus.cc:2229] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:40:03.188531 9972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:03.188621 10560 raft_consensus.cc:2258] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.189200 10561 ts_tablet_manager.cc:1905] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.190295 10562 ts_tablet_manager.cc:1905] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.190860 10560 ts_tablet_manager.cc:1905] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.193526 10561 ts_tablet_manager.cc:1918] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:03.193638 10561 log.cc:1196] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/b82a9ec2d2e04845bdcd6c4e1c15ded5
I20260808 10:40:03.193974 10561 ts_tablet_manager.cc:1939] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P b24a71873a1543ccad0413b817c7a22b: Deleting consensus metadata
I20260808 10:40:03.194447 10420 tablet_service.cc:1467] Processing CreateTablet for tablet e2b9f9d93d11409eb0875716a9bba480 (DEFAULT_TABLE table=test-table [id=374ef8e9232c47fab399f82b9e63a0f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.194710 10420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b9f9d93d11409eb0875716a9bba480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.194723 10270 tablet_service.cc:1467] Processing CreateTablet for tablet e2b9f9d93d11409eb0875716a9bba480 (DEFAULT_TABLE table=test-table [id=374ef8e9232c47fab399f82b9e63a0f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.194947 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b9f9d93d11409eb0875716a9bba480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.195183 9958 catalog_manager.cc:4841] TS b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647): tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 (table test-table [id=3c208e907b844593a2e3ab03c1e66465]) successfully deleted
I20260808 10:40:03.195596 10121 tablet_service.cc:1467] Processing CreateTablet for tablet e2b9f9d93d11409eb0875716a9bba480 (DEFAULT_TABLE table=test-table [id=374ef8e9232c47fab399f82b9e63a0f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.195816 10121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b9f9d93d11409eb0875716a9bba480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.197023 10527 tablet_bootstrap.cc:492] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Bootstrap starting.
I20260808 10:40:03.198071 10530 tablet_bootstrap.cc:492] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Bootstrap starting.
I20260808 10:40:03.198083 10527 tablet_bootstrap.cc:654] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.199126 10530 tablet_bootstrap.cc:654] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.199301 10527 tablet_bootstrap.cc:492] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: No bootstrap required, opened a new log
I20260808 10:40:03.199394 10527 ts_tablet_manager.cc:1397] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:03.199580 10562 ts_tablet_manager.cc:1918] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:03.199664 10560 ts_tablet_manager.cc:1918] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:03.199766 10560 log.cc:1196] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/b82a9ec2d2e04845bdcd6c4e1c15ded5
I20260808 10:40:03.199867 10527 raft_consensus.cc:348] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.199983 10527 raft_consensus.cc:374] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.200019 10527 raft_consensus.cc:729] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3cce12a2044c2a95327b261568d029, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.200079 10562 log.cc:1196] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/b82a9ec2d2e04845bdcd6c4e1c15ded5
I20260808 10:40:03.200142 10527 consensus_queue.cc:260] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.200381 10560 ts_tablet_manager.cc:1939] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P 5a3cce12a2044c2a95327b261568d029: Deleting consensus metadata
I20260808 10:40:03.200630 10562 ts_tablet_manager.cc:1939] T b82a9ec2d2e04845bdcd6c4e1c15ded5 P f6cd724bd644498396b3f12b7c9cc054: Deleting consensus metadata
I20260808 10:40:03.200397 10527 ts_tablet_manager.cc:1428] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.201119 10528 tablet_bootstrap.cc:492] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Bootstrap starting.
I20260808 10:40:03.202132 10528 tablet_bootstrap.cc:654] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.202335 9956 catalog_manager.cc:4841] TS f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065): tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 (table test-table [id=3c208e907b844593a2e3ab03c1e66465]) successfully deleted
I20260808 10:40:03.203179 9956 catalog_manager.cc:4841] TS 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): tablet b82a9ec2d2e04845bdcd6c4e1c15ded5 (table test-table [id=3c208e907b844593a2e3ab03c1e66465]) successfully deleted
I20260808 10:40:03.204137 10528 tablet_bootstrap.cc:492] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: No bootstrap required, opened a new log
I20260808 10:40:03.204140 10530 tablet_bootstrap.cc:492] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: No bootstrap required, opened a new log
I20260808 10:40:03.204239 10528 ts_tablet_manager.cc:1397] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:03.204269 10530 ts_tablet_manager.cc:1397] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 10:40:03.204700 10530 raft_consensus.cc:348] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.204700 10528 raft_consensus.cc:348] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.204823 10530 raft_consensus.cc:374] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.204823 10528 raft_consensus.cc:374] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.204867 10530 raft_consensus.cc:729] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.204867 10528 raft_consensus.cc:729] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.204993 10528 consensus_queue.cc:260] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.204993 10530 consensus_queue.cc:260] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.205248 10530 ts_tablet_manager.cc:1428] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.205265 10528 ts_tablet_manager.cc:1428] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.223373 10536 raft_consensus.cc:482] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:03.223527 10536 raft_consensus.cc:504] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.224207 10536 leader_election.cc:288] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.224635 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b9f9d93d11409eb0875716a9bba480" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029" is_pre_election: true
I20260808 10:40:03.224647 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b9f9d93d11409eb0875716a9bba480" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054" is_pre_election: true
I20260808 10:40:03.224792 10440 raft_consensus.cc:2454] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 0.
I20260808 10:40:03.224809 10293 raft_consensus.cc:2454] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 0.
I20260808 10:40:03.225116 10071 leader_election.cc:302] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.225302 10536 raft_consensus.cc:2781] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:03.225355 10536 raft_consensus.cc:482] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:03.225396 10536 raft_consensus.cc:3037] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.226385 10536 raft_consensus.cc:504] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.226701 10536 leader_election.cc:288] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Requested vote from peers f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.227128 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b9f9d93d11409eb0875716a9bba480" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029"
I20260808 10:40:03.227138 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b9f9d93d11409eb0875716a9bba480" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054"
I20260808 10:40:03.227252 10440 raft_consensus.cc:3037] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.227252 10293 raft_consensus.cc:3037] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.228418 10440 raft_consensus.cc:2454] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 1.
I20260808 10:40:03.228418 10293 raft_consensus.cc:2454] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 1.
I20260808 10:40:03.228704 10071 leader_election.cc:302] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.228971 10536 raft_consensus.cc:2781] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:03.229212 10536 raft_consensus.cc:686] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 1 LEADER]: Becoming Leader. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Running, Role: LEADER
I20260808 10:40:03.229466 10536 consensus_queue.cc:237] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.231964 9972 catalog_manager.cc:5495] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b reported cstate change: term changed from 0 to 1, leader changed from <none> to b24a71873a1543ccad0413b817c7a22b (127.1.194.65). New cstate: current_term: 1 leader_uuid: "b24a71873a1543ccad0413b817c7a22b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } health_report { overall_health: UNKNOWN } } }
W20260808 10:40:03.253172 10497 tablet.cc:2369] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:40:03.268383 10196 tablet.cc:2369] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185602, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.126@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:03.272783 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.263431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.1.194.126:46465)
0808 10:40:03.263579 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.263583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.263815 (+ 232us) server_negotiation.cc:407] Connection header received
0808 10:40:03.263929 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.263931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.263978 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.264054 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:03.264914 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.265551 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.266686 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.266889 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.266973 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.269746 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:03.269749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:03.269751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:03.269770 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:03.271111 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.271563 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.271568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.271570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.271642 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.271904 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.271910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.271912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.272081 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:03.272182 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.272479 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.272655 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":66}
I20260808 10:40:03.272783 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.263347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46465 (local address 127.0.0.1:51030)
0808 10:40:03.263679 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.263696 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.263831 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.264130 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.264133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.264652 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:03.264783 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.264790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.265666 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.265670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.266556 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.266568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.266990 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.266993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.267087 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.267775 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:03.267788 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:03.269607 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:03.271218 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:03.271229 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:03.271236 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:03.271460 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:03.271728 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:03.271732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:03.271734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:03.271822 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:03.272190 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:03.272199 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:03.272308 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.272435 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.272595 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":115,"threads_started":1}
I20260808 10:40:03.277025 9972 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:51030:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:40:03.277531 10345 tablet.cc:2369] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:40:03.277859 9972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:03.282676 10121 tablet_service.cc:1467] Processing CreateTablet for tablet 719476d776e648199ae3bad2426d4686 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c670fa6ab9a64b15856174b8ddd436cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:40:03.282868 10270 tablet_service.cc:1467] Processing CreateTablet for tablet 719476d776e648199ae3bad2426d4686 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c670fa6ab9a64b15856174b8ddd436cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:40:03.283118 10121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 719476d776e648199ae3bad2426d4686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.282994 10420 tablet_service.cc:1467] Processing CreateTablet for tablet 719476d776e648199ae3bad2426d4686 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c670fa6ab9a64b15856174b8ddd436cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:40:03.283236 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 719476d776e648199ae3bad2426d4686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.283331 10420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 719476d776e648199ae3bad2426d4686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.286151 10527 tablet_bootstrap.cc:492] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: Bootstrap starting.
I20260808 10:40:03.286151 10528 tablet_bootstrap.cc:492] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: Bootstrap starting.
I20260808 10:40:03.286226 10530 tablet_bootstrap.cc:492] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: Bootstrap starting.
I20260808 10:40:03.287135 10528 tablet_bootstrap.cc:654] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.287135 10527 tablet_bootstrap.cc:654] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.287201 10530 tablet_bootstrap.cc:654] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.288605 10528 tablet_bootstrap.cc:492] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: No bootstrap required, opened a new log
I20260808 10:40:03.288691 10528 ts_tablet_manager.cc:1397] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:03.289211 10528 raft_consensus.cc:348] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.289340 10528 raft_consensus.cc:374] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.289383 10528 raft_consensus.cc:729] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.289557 10530 tablet_bootstrap.cc:492] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: No bootstrap required, opened a new log
I20260808 10:40:03.289553 10528 consensus_queue.cc:260] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.289654 10530 ts_tablet_manager.cc:1397] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:03.289827 10528 ts_tablet_manager.cc:1428] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.290023 10536 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.290153 10536 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.290089 10530 raft_consensus.cc:348] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.290269 10530 raft_consensus.cc:374] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.290336 10530 raft_consensus.cc:729] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.290485 10530 consensus_queue.cc:260] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.291263 10527 tablet_bootstrap.cc:492] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: No bootstrap required, opened a new log
I20260808 10:40:03.291364 10527 ts_tablet_manager.cc:1397] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:40:03.291847 10527 raft_consensus.cc:348] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.291972 10527 raft_consensus.cc:374] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.291947 10543 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.292009 10527 raft_consensus.cc:729] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3cce12a2044c2a95327b261568d029, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.292074 10543 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.292147 10527 consensus_queue.cc:260] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.292426 10527 ts_tablet_manager.cc:1428] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.292796 10530 ts_tablet_manager.cc:1428] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:40:03.292729 10534 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.292881 10534 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.474296 10534 raft_consensus.cc:482] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:03.474472 10534 raft_consensus.cc:504] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.475759 10534 leader_election.cc:288] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647), f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065)
I20260808 10:40:03.480391 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.476033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44647 (local address 127.1.194.67:58937)
0808 10:40:03.476245 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.476262 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.476381 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.476759 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.476763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.476808 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.477044 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.477052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.478171 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.478173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.479274 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.479285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.479935 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.479938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.480044 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.480077 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.480148 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.480222 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":57,"spinlock_wait_cycles":10624}
I20260808 10:40:03.480602 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.476137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:45541 (local address 127.1.194.66:41065)
0808 10:40:03.476259 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.476264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.476452 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:40:03.476555 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.476558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.476613 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.476700 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.477170 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.478005 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.479437 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.479893 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.479995 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.480052 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.480389 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.480478 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:40:03.480618 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.476033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:41065 (local address 127.1.194.67:45541)
0808 10:40:03.476343 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.476355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.476459 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.476759 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.476763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.476807 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:03.477044 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.477052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.478159 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.478162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.479287 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.479298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.480163 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.480166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.480273 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.480290 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.480356 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.480405 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":113,"spinlock_wait_cycles":27904,"thread_start_us":70,"threads_started":1}
I20260808 10:40:03.480795 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.476137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:58937 (local address 127.1.194.65:44647)
0808 10:40:03.476273 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.476279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.476327 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:40:03.476533 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.476536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.476629 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.476704 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:03.477170 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.478005 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.479426 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.479820 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.479943 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.479998 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.480591 (+ 593us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.480673 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:40:03.481010 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719476d776e648199ae3bad2426d4686" candidate_uuid: "5a3cce12a2044c2a95327b261568d029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054" is_pre_election: true
I20260808 10:40:03.481184 10293 raft_consensus.cc:2454] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3cce12a2044c2a95327b261568d029 in term 0.
I20260808 10:40:03.481199 10142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719476d776e648199ae3bad2426d4686" candidate_uuid: "5a3cce12a2044c2a95327b261568d029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24a71873a1543ccad0413b817c7a22b" is_pre_election: true
I20260808 10:40:03.481323 10142 raft_consensus.cc:2454] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3cce12a2044c2a95327b261568d029 in term 0.
I20260808 10:40:03.481598 10370 leader_election.cc:302] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3cce12a2044c2a95327b261568d029, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.481920 10534 raft_consensus.cc:2781] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:03.481999 10534 raft_consensus.cc:482] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:03.482030 10534 raft_consensus.cc:3037] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.483080 10534 raft_consensus.cc:504] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.483424 10534 leader_election.cc:288] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [CANDIDATE]: Term 1 election: Requested vote from peers b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647), f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065)
I20260808 10:40:03.483803 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719476d776e648199ae3bad2426d4686" candidate_uuid: "5a3cce12a2044c2a95327b261568d029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054"
I20260808 10:40:03.483927 10293 raft_consensus.cc:3037] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.484833 10293 raft_consensus.cc:2454] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3cce12a2044c2a95327b261568d029 in term 1.
I20260808 10:40:03.485114 10142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719476d776e648199ae3bad2426d4686" candidate_uuid: "5a3cce12a2044c2a95327b261568d029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24a71873a1543ccad0413b817c7a22b"
I20260808 10:40:03.485146 10370 leader_election.cc:302] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3cce12a2044c2a95327b261568d029, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.485256 10142 raft_consensus.cc:3037] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.485363 10534 raft_consensus.cc:2781] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:03.485618 10534 raft_consensus.cc:686] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [term 1 LEADER]: Becoming Leader. State: Replica: 5a3cce12a2044c2a95327b261568d029, State: Running, Role: LEADER
I20260808 10:40:03.485890 10534 consensus_queue.cc:237] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.486526 10142 raft_consensus.cc:2454] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3cce12a2044c2a95327b261568d029 in term 1.
I20260808 10:40:03.487272 10580 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: TxnStatusTablet state changed. Reason: New leader 5a3cce12a2044c2a95327b261568d029. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.487407 10580 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:40:03.487681 10582 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:40:03.487851 10582 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:40:03.488376 9971 catalog_manager.cc:5495] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a3cce12a2044c2a95327b261568d029 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } health_report { overall_health: HEALTHY } } }
I20260808 10:40:03.536218 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.531459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36651 (local address 127.0.0.1:39550)
0808 10:40:03.531860 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.531875 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.531987 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.532328 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.532331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.532339 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:40:03.532453 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.532459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.533324 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.533327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.534337 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.534348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.534969 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.534974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.535068 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.535114 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.535931 (+ 817us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.535948 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.536021 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.536107 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295}
I20260808 10:40:03.536269 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.531693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.1.194.67:36651)
0808 10:40:03.531830 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.531835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.531932 (+ 97us) server_negotiation.cc:407] Connection header received
0808 10:40:03.532134 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.532137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.532199 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.532302 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.532567 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.533210 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.534480 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.534845 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.534946 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.535298 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.535826 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.535969 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.536033 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.536112 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:40:03.539427 10142 raft_consensus.cc:1261] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Refusing update from remote peer 5a3cce12a2044c2a95327b261568d029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.539598 10293 raft_consensus.cc:1261] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Refusing update from remote peer 5a3cce12a2044c2a95327b261568d029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.540005 10580 consensus_queue.cc:1035] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.540246 10534 consensus_queue.cc:1035] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.542129 10543 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: TxnStatusTablet state changed. Reason: New leader 5a3cce12a2044c2a95327b261568d029. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.542292 10543 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.542454 10536 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: TxnStatusTablet state changed. Reason: New leader 5a3cce12a2044c2a95327b261568d029. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.542567 10536 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.544536 10534 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.544657 10534 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:40:03.545035 10543 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.545143 10543 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P f6cd724bd644498396b3f12b7c9cc054: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.546571 10534 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.546717 10534 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P 5a3cce12a2044c2a95327b261568d029: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:40:03.547341 10551 mvcc.cc:205] Tried to move back new op lower bound from 7316216232092135424 to 7316216231881338880. Current Snapshot: MvccSnapshot[applied={T|T < 7316216232092135424}]
I20260808 10:40:03.547539 10536 tablet_replica.cc:439] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a3cce12a2044c2a95327b261568d029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } } }
I20260808 10:40:03.547640 10536 tablet_replica.cc:442] T 719476d776e648199ae3bad2426d4686 P b24a71873a1543ccad0413b817c7a22b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:40:03.555281 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.551198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53662 (local address 127.1.194.65:44647)
0808 10:40:03.551333 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.551338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.551357 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:40:03.551542 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.551546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.551611 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.551725 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.551964 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.552732 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.553650 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.553997 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.554108 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.554317 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.554867 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.554982 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.555085 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.555145 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:40:03.565773 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.562147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36651 (local address 127.1.194.65:47027)
0808 10:40:03.562452 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.562465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.562553 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.562796 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.562798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.562804 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:40:03.562900 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.562904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.563863 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.563866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.564487 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.564495 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.565059 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.565062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.565133 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.565174 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.565448 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.565452 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.565562 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.565640 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":104,"threads_started":1}
I20260808 10:40:03.565768 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.562241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:47027 (local address 127.1.194.67:36651)
0808 10:40:03.562365 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.562369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.562528 (+ 159us) server_negotiation.cc:407] Connection header received
0808 10:40:03.562645 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.562649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.562700 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.562779 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.562995 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.563754 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.564606 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.564941 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.565042 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.565290 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.565367 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.565436 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.565574 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.565629 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:40:03.573244 10293 raft_consensus.cc:1261] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Refusing update from remote peer b24a71873a1543ccad0413b817c7a22b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.573261 10440 raft_consensus.cc:1261] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Refusing update from remote peer b24a71873a1543ccad0413b817c7a22b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.573818 10567 consensus_queue.cc:1035] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.574034 10536 consensus_queue.cc:1035] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.599050 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.594288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44647 (local address 127.1.194.67:33719)
0808 10:40:03.594967 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.594984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:03.595135 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:03.595626 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:03.595630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:03.595642 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:40:03.595790 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.595799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.596832 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.596835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.597599 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:03.597617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.598153 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.598157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:03.598258 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.598314 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.598722 (+ 408us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.598730 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:40:03.598796 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:40:03.598888 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":108,"threads_started":1}
I20260808 10:40:03.599035 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.594678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:33719 (local address 127.1.194.65:44647)
0808 10:40:03.594800 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.594805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.595076 (+ 271us) server_negotiation.cc:407] Connection header received
0808 10:40:03.595297 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.595301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.595383 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.595484 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.595950 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.596692 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.597739 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.598006 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.598151 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.598496 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.598609 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.598693 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.598816 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.598876 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:40:03.626267 9971 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.626464 9971 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.628810 9971 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Sending DeleteTablet for 3 replicas of tablet e2b9f9d93d11409eb0875716a9bba480
I20260808 10:40:03.629510 10270 tablet_service.cc:1514] Processing DeleteTablet for tablet e2b9f9d93d11409eb0875716a9bba480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:49256
I20260808 10:40:03.629617 10121 tablet_service.cc:1514] Processing DeleteTablet for tablet e2b9f9d93d11409eb0875716a9bba480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:53650
I20260808 10:40:03.629714 10562 tablet_replica.cc:330] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: stopping tablet replica
I20260808 10:40:03.629822 10561 tablet_replica.cc:330] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: stopping tablet replica
I20260808 10:40:03.629863 10562 raft_consensus.cc:2229] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:03.629930 10561 raft_consensus.cc:2229] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:40:03.629988 10562 raft_consensus.cc:2258] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.630195 10561 raft_consensus.cc:2258] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.630966 10561 ts_tablet_manager.cc:1905] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.631001 10562 ts_tablet_manager.cc:1905] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.633900 10420 tablet_service.cc:1514] Processing DeleteTablet for tablet e2b9f9d93d11409eb0875716a9bba480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39542
I20260808 10:40:03.634106 10560 tablet_replica.cc:330] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: stopping tablet replica
I20260808 10:40:03.634240 10560 raft_consensus.cc:2229] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:03.634299 10561 ts_tablet_manager.cc:1918] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:40:03.634361 10560 raft_consensus.cc:2258] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.634371 10561 log.cc:1196] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/e2b9f9d93d11409eb0875716a9bba480
I20260808 10:40:03.634742 10561 ts_tablet_manager.cc:1939] T e2b9f9d93d11409eb0875716a9bba480 P b24a71873a1543ccad0413b817c7a22b: Deleting consensus metadata
I20260808 10:40:03.635165 10560 ts_tablet_manager.cc:1905] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.635456 9958 catalog_manager.cc:4841] TS b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647): tablet e2b9f9d93d11409eb0875716a9bba480 (table test-table [id=374ef8e9232c47fab399f82b9e63a0f2]) successfully deleted
I20260808 10:40:03.638062 10560 ts_tablet_manager.cc:1918] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:40:03.638154 10560 log.cc:1196] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/e2b9f9d93d11409eb0875716a9bba480
I20260808 10:40:03.638428 10560 ts_tablet_manager.cc:1939] T e2b9f9d93d11409eb0875716a9bba480 P 5a3cce12a2044c2a95327b261568d029: Deleting consensus metadata
I20260808 10:40:03.638827 10562 ts_tablet_manager.cc:1918] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:40:03.638975 10562 log.cc:1196] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/e2b9f9d93d11409eb0875716a9bba480
I20260808 10:40:03.639211 9956 catalog_manager.cc:4841] TS 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): tablet e2b9f9d93d11409eb0875716a9bba480 (table test-table [id=374ef8e9232c47fab399f82b9e63a0f2]) successfully deleted
I20260808 10:40:03.639290 10562 ts_tablet_manager.cc:1939] T e2b9f9d93d11409eb0875716a9bba480 P f6cd724bd644498396b3f12b7c9cc054: Deleting consensus metadata
I20260808 10:40:03.640100 9956 catalog_manager.cc:4841] TS f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065): tablet e2b9f9d93d11409eb0875716a9bba480 (table test-table [id=374ef8e9232c47fab399f82b9e63a0f2]) successfully deleted
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:40:03.673046 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:03.665321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.1.194.126:46465)
0808 10:40:03.665468 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.665473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.665784 (+ 311us) server_negotiation.cc:407] Connection header received
0808 10:40:03.665918 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.665921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.665982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.666075 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:03.666933 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.667715 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.668958 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.669263 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.669363 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.672917 (+ 3554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51032: BlockingRecv error: recv got EOF from 127.0.0.1:51032 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[9924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM
I20260808 10:40:03.684087 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.674808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.1.194.126:46465)
0808 10:40:03.674923 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.674927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.675093 (+ 166us) server_negotiation.cc:407] Connection header received
0808 10:40:03.675198 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.675201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.675248 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.675347 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:03.675951 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.676753 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.677548 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.677890 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.677995 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.680714 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:03.680718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:03.680722 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:03.680753 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:03.682401 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.682808 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.682811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.682814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.682900 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:03.683151 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:03.683154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:03.683156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:03.683454 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:03.683601 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.683805 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.683921 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:40:03.686968 9971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:40:03.687423 9971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:03.692524 10121 tablet_service.cc:1467] Processing CreateTablet for tablet b72b69a47ea249508b1d61e84a4ad917 (DEFAULT_TABLE table=test-table [id=fc61b5c3358a4a2fb80707b7666475ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.692548 10270 tablet_service.cc:1467] Processing CreateTablet for tablet b72b69a47ea249508b1d61e84a4ad917 (DEFAULT_TABLE table=test-table [id=fc61b5c3358a4a2fb80707b7666475ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.692850 10121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72b69a47ea249508b1d61e84a4ad917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.692847 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72b69a47ea249508b1d61e84a4ad917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.694826 10528 tablet_bootstrap.cc:492] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Bootstrap starting.
I20260808 10:40:03.695214 10530 tablet_bootstrap.cc:492] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Bootstrap starting.
I20260808 10:40:03.695367 10420 tablet_service.cc:1467] Processing CreateTablet for tablet b72b69a47ea249508b1d61e84a4ad917 (DEFAULT_TABLE table=test-table [id=fc61b5c3358a4a2fb80707b7666475ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.695657 10420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72b69a47ea249508b1d61e84a4ad917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.695787 10528 tablet_bootstrap.cc:654] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.696002 10530 tablet_bootstrap.cc:654] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.697412 10530 tablet_bootstrap.cc:492] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: No bootstrap required, opened a new log
I20260808 10:40:03.697531 10530 ts_tablet_manager.cc:1397] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:03.697912 10530 raft_consensus.cc:348] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.698022 10530 raft_consensus.cc:374] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.698010 10527 tablet_bootstrap.cc:492] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Bootstrap starting.
I20260808 10:40:03.698056 10530 raft_consensus.cc:729] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.698156 10530 consensus_queue.cc:260] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.698396 10530 ts_tablet_manager.cc:1428] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.698889 10527 tablet_bootstrap.cc:654] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.699252 10528 tablet_bootstrap.cc:492] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: No bootstrap required, opened a new log
I20260808 10:40:03.699331 10528 ts_tablet_manager.cc:1397] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:40:03.699810 10528 raft_consensus.cc:348] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.699970 10528 raft_consensus.cc:374] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.700007 10528 raft_consensus.cc:729] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.700124 10528 consensus_queue.cc:260] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.700418 10528 ts_tablet_manager.cc:1428] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.702222 10527 tablet_bootstrap.cc:492] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: No bootstrap required, opened a new log
I20260808 10:40:03.702327 10527 ts_tablet_manager.cc:1397] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:40:03.702761 10527 raft_consensus.cc:348] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.702874 10527 raft_consensus.cc:374] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.702924 10527 raft_consensus.cc:729] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3cce12a2044c2a95327b261568d029, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.703042 10527 consensus_queue.cc:260] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.703312 10527 ts_tablet_manager.cc:1428] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.728554 10543 raft_consensus.cc:482] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:03.728718 10543 raft_consensus.cc:504] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.729142 10543 leader_election.cc:288] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.729580 10142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72b69a47ea249508b1d61e84a4ad917" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24a71873a1543ccad0413b817c7a22b" is_pre_election: true
I20260808 10:40:03.729596 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72b69a47ea249508b1d61e84a4ad917" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029" is_pre_election: true
I20260808 10:40:03.729738 10142 raft_consensus.cc:2454] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 0.
I20260808 10:40:03.729758 10440 raft_consensus.cc:2454] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 0.
I20260808 10:40:03.730015 10223 leader_election.cc:302] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.730180 10543 raft_consensus.cc:2781] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:03.730252 10543 raft_consensus.cc:482] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:03.730316 10543 raft_consensus.cc:3037] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.731369 10543 raft_consensus.cc:504] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.731717 10543 leader_election.cc:288] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 election: Requested vote from peers b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.732018 10142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72b69a47ea249508b1d61e84a4ad917" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24a71873a1543ccad0413b817c7a22b"
I20260808 10:40:03.732141 10142 raft_consensus.cc:3037] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.732067 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72b69a47ea249508b1d61e84a4ad917" candidate_uuid: "f6cd724bd644498396b3f12b7c9cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029"
I20260808 10:40:03.732240 10440 raft_consensus.cc:3037] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.733150 10440 raft_consensus.cc:2454] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 1.
I20260808 10:40:03.733256 10142 raft_consensus.cc:2454] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cd724bd644498396b3f12b7c9cc054 in term 1.
I20260808 10:40:03.733426 10221 leader_election.cc:302] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3cce12a2044c2a95327b261568d029, f6cd724bd644498396b3f12b7c9cc054; no voters:
I20260808 10:40:03.733633 10543 raft_consensus.cc:2781] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:03.733825 10543 raft_consensus.cc:686] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 1 LEADER]: Becoming Leader. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Running, Role: LEADER
I20260808 10:40:03.733968 10543 consensus_queue.cc:237] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.735744 9971 catalog_manager.cc:5495] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "f6cd724bd644498396b3f12b7c9cc054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } health_report { overall_health: UNKNOWN } } }
I20260808 10:40:03.754303 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.751214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.1.194.66:41065)
0808 10:40:03.751347 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.751351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.751366 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:03.751465 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.751467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.751519 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.751604 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.751855 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.752368 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.753230 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.753453 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.753607 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.753860 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.753937 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.754012 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.754109 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.754160 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:40:03.756587 10142 raft_consensus.cc:1261] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Refusing update from remote peer f6cd724bd644498396b3f12b7c9cc054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.756590 10440 raft_consensus.cc:1261] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Refusing update from remote peer f6cd724bd644498396b3f12b7c9cc054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.757007 10543 consensus_queue.cc:1035] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.757167 10619 consensus_queue.cc:1035] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.764895 9973 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.765215 9973 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.767577 9973 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Sending DeleteTablet for 3 replicas of tablet b72b69a47ea249508b1d61e84a4ad917
I20260808 10:40:03.768195 10121 tablet_service.cc:1514] Processing DeleteTablet for tablet b72b69a47ea249508b1d61e84a4ad917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:53650
I20260808 10:40:03.768512 10561 tablet_replica.cc:330] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: stopping tablet replica
I20260808 10:40:03.768613 10419 tablet_service.cc:1514] Processing DeleteTablet for tablet b72b69a47ea249508b1d61e84a4ad917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39542
I20260808 10:40:03.768652 10561 raft_consensus.cc:2229] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:03.768786 10561 raft_consensus.cc:2258] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.768903 9973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:40:03.768957 10420 tablet_service.cc:1514] Processing DeleteTablet for tablet b72b69a47ea249508b1d61e84a4ad917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39542
I20260808 10:40:03.769148 10560 tablet_replica.cc:330] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: stopping tablet replica
I20260808 10:40:03.769263 10560 raft_consensus.cc:2229] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:40:03.769302 9973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:03.769395 10560 raft_consensus.cc:2258] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.769722 10270 tablet_service.cc:1514] Processing DeleteTablet for tablet b72b69a47ea249508b1d61e84a4ad917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:49256
I20260808 10:40:03.769891 10562 tablet_replica.cc:330] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: stopping tablet replica
I20260808 10:40:03.770015 10562 raft_consensus.cc:2229] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:40:03.770190 10562 raft_consensus.cc:2258] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.770529 10561 ts_tablet_manager.cc:1905] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.770799 10560 ts_tablet_manager.cc:1905] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.772930 10561 ts_tablet_manager.cc:1918] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:03.773016 10561 log.cc:1196] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/b72b69a47ea249508b1d61e84a4ad917
I20260808 10:40:03.773380 10560 ts_tablet_manager.cc:1918] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:03.773718 10561 ts_tablet_manager.cc:1939] T b72b69a47ea249508b1d61e84a4ad917 P b24a71873a1543ccad0413b817c7a22b: Deleting consensus metadata
I20260808 10:40:03.773454 10560 log.cc:1196] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/b72b69a47ea249508b1d61e84a4ad917
I20260808 10:40:03.774425 9958 catalog_manager.cc:4841] TS b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647): tablet b72b69a47ea249508b1d61e84a4ad917 (table test-table [id=fc61b5c3358a4a2fb80707b7666475ee]) successfully deleted
I20260808 10:40:03.777324 10121 tablet_service.cc:1467] Processing CreateTablet for tablet 2c179664a42448d8be29e56a9c1fc8a4 (DEFAULT_TABLE table=test-table [id=ce6f7d6d38bd4049ab550e80316b046e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.777633 10121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c179664a42448d8be29e56a9c1fc8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.777779 10270 tablet_service.cc:1467] Processing CreateTablet for tablet 2c179664a42448d8be29e56a9c1fc8a4 (DEFAULT_TABLE table=test-table [id=ce6f7d6d38bd4049ab550e80316b046e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.778069 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c179664a42448d8be29e56a9c1fc8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.781123 10528 tablet_bootstrap.cc:492] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: Bootstrap starting.
I20260808 10:40:03.781648 10530 tablet_bootstrap.cc:492] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Bootstrap starting.
I20260808 10:40:03.782143 10528 tablet_bootstrap.cc:654] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.782514 10530 tablet_bootstrap.cc:654] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.783411 10528 tablet_bootstrap.cc:492] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: No bootstrap required, opened a new log
I20260808 10:40:03.783494 10528 ts_tablet_manager.cc:1397] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:40:03.783901 10528 raft_consensus.cc:348] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.784011 10528 raft_consensus.cc:374] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.784051 10528 raft_consensus.cc:729] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.784165 10528 consensus_queue.cc:260] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.784399 10528 ts_tablet_manager.cc:1428] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:40:03.779991 10420 tablet_service.cc:1467] Processing CreateTablet for tablet 2c179664a42448d8be29e56a9c1fc8a4 (DEFAULT_TABLE table=test-table [id=ce6f7d6d38bd4049ab550e80316b046e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:03.785553 10530 tablet_bootstrap.cc:492] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: No bootstrap required, opened a new log
I20260808 10:40:03.785629 10530 ts_tablet_manager.cc:1397] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:40:03.785740 10420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c179664a42448d8be29e56a9c1fc8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:03.786012 10530 raft_consensus.cc:348] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.786142 10530 raft_consensus.cc:374] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.786183 10530 raft_consensus.cc:729] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cd724bd644498396b3f12b7c9cc054, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.786301 10530 consensus_queue.cc:260] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.786533 10530 ts_tablet_manager.cc:1428] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:40:03.787966 10527 tablet_bootstrap.cc:492] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Bootstrap starting.
I20260808 10:40:03.788811 10527 tablet_bootstrap.cc:654] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:03.774551 10560 ts_tablet_manager.cc:1939] T b72b69a47ea249508b1d61e84a4ad917 P 5a3cce12a2044c2a95327b261568d029: Deleting consensus metadata
I20260808 10:40:03.790405 9956 catalog_manager.cc:4841] TS 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): tablet b72b69a47ea249508b1d61e84a4ad917 (table test-table [id=fc61b5c3358a4a2fb80707b7666475ee]) successfully deleted
W20260808 10:40:03.790720 9956 catalog_manager.cc:4808] TS 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): delete failed for tablet b72b69a47ea249508b1d61e84a4ad917 because the tablet was not found. No further retry: Not found: Tablet not found: b72b69a47ea249508b1d61e84a4ad917
I20260808 10:40:03.794821 10567 raft_consensus.cc:482] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:03.794929 10567 raft_consensus.cc:504] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.795312 10567 leader_election.cc:288] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.795670 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c179664a42448d8be29e56a9c1fc8a4" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029" is_pre_election: true
I20260808 10:40:03.796113 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c179664a42448d8be29e56a9c1fc8a4" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054" is_pre_election: true
I20260808 10:40:03.796226 10293 raft_consensus.cc:2454] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 0.
W20260808 10:40:03.796262 10071 leader_election.cc:341] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): Illegal state: must be running to vote when last-logged opid is not known
I20260808 10:40:03.796434 10071 leader_election.cc:302] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b, f6cd724bd644498396b3f12b7c9cc054; no voters: 5a3cce12a2044c2a95327b261568d029
I20260808 10:40:03.796607 10567 raft_consensus.cc:2781] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:03.796672 10567 raft_consensus.cc:482] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:03.796706 10567 raft_consensus.cc:3037] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.797940 10567 raft_consensus.cc:504] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.798128 10562 ts_tablet_manager.cc:1905] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.798244 10567 leader_election.cc:288] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Requested vote from peers f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065), 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651)
I20260808 10:40:03.798743 10440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c179664a42448d8be29e56a9c1fc8a4" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3cce12a2044c2a95327b261568d029"
I20260808 10:40:03.798972 10293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c179664a42448d8be29e56a9c1fc8a4" candidate_uuid: "b24a71873a1543ccad0413b817c7a22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cd724bd644498396b3f12b7c9cc054"
I20260808 10:40:03.799064 10293 raft_consensus.cc:3037] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.800079 10293 raft_consensus.cc:2454] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24a71873a1543ccad0413b817c7a22b in term 1.
I20260808 10:40:03.800568 10562 ts_tablet_manager.cc:1918] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260808 10:40:03.800566 10071 leader_election.cc:341] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): Illegal state: must be running to vote when last-logged opid is not known
I20260808 10:40:03.800637 10562 log.cc:1196] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/b72b69a47ea249508b1d61e84a4ad917
I20260808 10:40:03.800668 10071 leader_election.cc:302] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24a71873a1543ccad0413b817c7a22b, f6cd724bd644498396b3f12b7c9cc054; no voters: 5a3cce12a2044c2a95327b261568d029
I20260808 10:40:03.800885 10562 ts_tablet_manager.cc:1939] T b72b69a47ea249508b1d61e84a4ad917 P f6cd724bd644498396b3f12b7c9cc054: Deleting consensus metadata
I20260808 10:40:03.801065 10567 raft_consensus.cc:2781] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:03.801148 10567 raft_consensus.cc:686] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 1 LEADER]: Becoming Leader. State: Replica: b24a71873a1543ccad0413b817c7a22b, State: Running, Role: LEADER
I20260808 10:40:03.801258 10567 consensus_queue.cc:237] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.801733 9956 catalog_manager.cc:4841] TS f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065): tablet b72b69a47ea249508b1d61e84a4ad917 (table test-table [id=fc61b5c3358a4a2fb80707b7666475ee]) successfully deleted
I20260808 10:40:03.802951 9972 catalog_manager.cc:5495] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b reported cstate change: term changed from 0 to 1, leader changed from <none> to b24a71873a1543ccad0413b817c7a22b (127.1.194.65). New cstate: current_term: 1 leader_uuid: "b24a71873a1543ccad0413b817c7a22b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } health_report { overall_health: UNKNOWN } } }
I20260808 10:40:03.806386 10527 tablet_bootstrap.cc:492] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: No bootstrap required, opened a new log
I20260808 10:40:03.806505 10527 ts_tablet_manager.cc:1397] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20260808 10:40:03.807101 10527 raft_consensus.cc:348] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.807250 10527 raft_consensus.cc:374] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:03.807289 10527 raft_consensus.cc:729] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3cce12a2044c2a95327b261568d029, State: Initialized, Role: FOLLOWER
I20260808 10:40:03.807415 10527 consensus_queue.cc:260] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24a71873a1543ccad0413b817c7a22b" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44647 } } peers { permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 } } peers { permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 } }
I20260808 10:40:03.807689 10527 ts_tablet_manager.cc:1428] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:40:03.821110 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.818073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.1.194.65:44647)
0808 10:40:03.818186 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:03.818191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:03.818205 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:03.818292 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:03.818295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:03.818340 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:03.818423 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:03.818615 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.819275 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.820052 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:03.820338 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:03.820440 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:03.820634 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:03.820713 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:03.820785 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:03.820914 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:40:03.820965 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:40:03.826817 10293 raft_consensus.cc:1261] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Refusing update from remote peer b24a71873a1543ccad0413b817c7a22b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.826828 10440 raft_consensus.cc:3037] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:03.827185 10567 consensus_queue.cc:1035] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cd724bd644498396b3f12b7c9cc054" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.828025 10440 raft_consensus.cc:1261] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Refusing update from remote peer b24a71873a1543ccad0413b817c7a22b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:03.828377 10567 consensus_queue.cc:1035] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3cce12a2044c2a95327b261568d029" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:03.859578 9971 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.859774 9971 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:03.861742 9971 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc4037197b0a4b409009d5b511cae8ca: Sending DeleteTablet for 3 replicas of tablet 2c179664a42448d8be29e56a9c1fc8a4
I20260808 10:40:03.862273 10121 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c179664a42448d8be29e56a9c1fc8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:53650
I20260808 10:40:03.862381 10270 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c179664a42448d8be29e56a9c1fc8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:49256
I20260808 10:40:03.862426 10420 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c179664a42448d8be29e56a9c1fc8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:03 UTC) from {username='oryx', principal='oryx/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39542
I20260808 10:40:03.862483 10561 tablet_replica.cc:330] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: stopping tablet replica
I20260808 10:40:03.862599 10561 raft_consensus.cc:2229] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:40:03.862613 10562 tablet_replica.cc:330] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: stopping tablet replica
I20260808 10:40:03.862627 10560 tablet_replica.cc:330] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: stopping tablet replica
I20260808 10:40:03.862721 10560 raft_consensus.cc:2229] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:03.862728 10562 raft_consensus.cc:2229] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:03.862793 10561 raft_consensus.cc:2258] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.862816 10560 raft_consensus.cc:2258] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.862843 10562 raft_consensus.cc:2258] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:03.863409 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 10053
I20260808 10:40:03.863448 10560 ts_tablet_manager.cc:1905] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.863497 10561 ts_tablet_manager.cc:1905] T 2c179664a42448d8be29e56a9c1fc8a4 P b24a71873a1543ccad0413b817c7a22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.866457 10562 ts_tablet_manager.cc:1905] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:03.866772 10560 ts_tablet_manager.cc:1918] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:40:03.866840 10560 log.cc:1196] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/2c179664a42448d8be29e56a9c1fc8a4
I20260808 10:40:03.867079 10560 ts_tablet_manager.cc:1939] T 2c179664a42448d8be29e56a9c1fc8a4 P 5a3cce12a2044c2a95327b261568d029: Deleting consensus metadata
I20260808 10:40:03.867905 9956 catalog_manager.cc:4841] TS 5a3cce12a2044c2a95327b261568d029 (127.1.194.67:36651): tablet 2c179664a42448d8be29e56a9c1fc8a4 (table test-table [id=ce6f7d6d38bd4049ab550e80316b046e]) successfully deleted
I20260808 10:40:03.868702 10562 ts_tablet_manager.cc:1918] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:40:03.868767 10562 log.cc:1196] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/2c179664a42448d8be29e56a9c1fc8a4
I20260808 10:40:03.868947 10562 ts_tablet_manager.cc:1939] T 2c179664a42448d8be29e56a9c1fc8a4 P f6cd724bd644498396b3f12b7c9cc054: Deleting consensus metadata
I20260808 10:40:03.869757 9956 catalog_manager.cc:4841] TS f6cd724bd644498396b3f12b7c9cc054 (127.1.194.66:41065): tablet 2c179664a42448d8be29e56a9c1fc8a4 (table test-table [id=ce6f7d6d38bd4049ab550e80316b046e]) successfully deleted
W20260808 10:40:03.875200 9958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:44647 (error 108)
I20260808 10:40:03.875382 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 10204
I20260808 10:40:03.876920 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:03.876005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44647 (local address 127.0.0.1:53692)
0808 10:40:03.876737 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:03.876817 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":111,"threads_started":1}
W20260808 10:40:03.877357 9958 catalog_manager.cc:4584] TS b24a71873a1543ccad0413b817c7a22b (127.1.194.65:44647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c179664a42448d8be29e56a9c1fc8a4: Network error: Client connection negotiation failed: client connection to 127.1.194.65:44647: connect: Connection refused (error 111)
I20260808 10:40:03.883874 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 10353
I20260808 10:40:03.892369 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 9937
2026-08-08T10:40:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3104 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[10635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[10635](info): set up 2 sockets
Aug 08 10:40:03 dist-test-slave-ss4n krb5kdc[10635](info): commencing operation
krb5kdc: starting...
W20260808 10:40:05.016156 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:40:05 dist-test-slave-ss4n krb5kdc[10635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:40:05.148890 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:33747
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:33747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:05.259764 10649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:05.260211 10649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:05.260320 10649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:05.265986 10649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:05.266065 10649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:05.266091 10649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:05.266115 10649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:05.266142 10649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:05.271960 10649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:33747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:33747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:05.272986 10649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:05.274081 10649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:05.282603 10655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:05.282631 10656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:05.282631 10658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:05.282745 10649 server_base.cc:1034] running on GCE node
I20260808 10:40:05.283380 10649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:05.284502 10649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:05.285686 10649 hybrid_clock.cc:648] HybridClock initialized: now 1786185605285665 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T10:40:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1399 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:40:05 dist-test-slave-ss4n krb5kdc[10664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:40:05 dist-test-slave-ss4n krb5kdc[10664](info): set up 2 sockets
Aug 08 10:40:05 dist-test-slave-ss4n krb5kdc[10664](info): commencing operation
krb5kdc: starting...
W20260808 10:40:06.412146 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:40:06 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:40:06.544816 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42553
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:06.655690 10677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:06.656016 10677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:06.656090 10677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:06.660132 10677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:06.660236 10677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:06.660274 10677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:06.660310 10677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:06.660346 10677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:06.665709 10677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:06.666811 10677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:06.667840 10677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:06.676335 10686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:06.676388 10683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:06.676362 10684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:06.676680 10677 server_base.cc:1034] running on GCE node
I20260808 10:40:06.677310 10677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:06.678485 10677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:06.679661 10677 hybrid_clock.cc:648] HybridClock initialized: now 1786185606679649 us; error 28 us; skew 500 ppm
Aug 08 10:40:06 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185606, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:06.682740 10677 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:40:06.683604 10677 webserver.cc:460] Webserver started at http://127.1.194.126:33057/ using document root <none> and password file <none>
I20260808 10:40:06.684118 10677 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:06.684177 10677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:06.684335 10677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:06.686676 10677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "dde49d6cb9714df1855843b9d8dfc90f"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "35fcb8c877c478f28e68f2fa47377d2b"
server_key_iv: "f8d40280c74766beb6bde1dc275189c8"
server_key_version: "encryptionkey@0"
I20260808 10:40:06.687232 10677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "dde49d6cb9714df1855843b9d8dfc90f"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "35fcb8c877c478f28e68f2fa47377d2b"
server_key_iv: "f8d40280c74766beb6bde1dc275189c8"
server_key_version: "encryptionkey@0"
I20260808 10:40:06.691123 10677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:40:06.693645 10695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:06.694998 10677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:40:06.695140 10677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "dde49d6cb9714df1855843b9d8dfc90f"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "35fcb8c877c478f28e68f2fa47377d2b"
server_key_iv: "f8d40280c74766beb6bde1dc275189c8"
server_key_version: "encryptionkey@0"
I20260808 10:40:06.695255 10677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:06.707319 10677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:06.710554 10677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:06.710767 10677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:06.718286 10677 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:42553
I20260808 10:40:06.718308 10752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:42553 every 8 connection(s)
I20260808 10:40:06.719321 10677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:40:06.722033 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 10677
I20260808 10:40:06.722198 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:40:06.722533 10753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:06.722549 1801 external_mini_cluster.cc:1428] Setting key 1fd692e25dee52d8a442d8d06d1d5701
I20260808 10:40:06.729349 10753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Bootstrap starting.
Aug 08 10:40:06 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:40:06.732199 10753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:06.733119 10753 log.cc:824] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:06.735507 10753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: No bootstrap required, opened a new log
I20260808 10:40:06.737883 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:06.724060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.1.194.126:42553)
0808 10:40:06.724518 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:06.724533 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:06.724567 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:40:06.725391 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:06.725408 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:06.725821 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:06.726142 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:06.726964 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:06.727932 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:06.728780 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:06.729243 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:06.729412 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:06.732371 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:06.732377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:06.732390 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:06.732417 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:06.734575 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:06.735014 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:06.735021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:06.735027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:06.735181 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:06.735465 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:06.735469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:06.735471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:06.736014 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:06.736229 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:06.736473 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 10:40:06.736785 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":154,"threads_started":1}
I20260808 10:40:06.740366 10753 raft_consensus.cc:348] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:06.740597 10753 raft_consensus.cc:374] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:06.740636 10753 raft_consensus.cc:729] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde49d6cb9714df1855843b9d8dfc90f, State: Initialized, Role: FOLLOWER
I20260808 10:40:06.741055 10753 consensus_queue.cc:260] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:06.741197 10753 raft_consensus.cc:388] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:40:06.741241 10753 raft_consensus.cc:482] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:40:06.741318 10753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:06.742347 10753 raft_consensus.cc:504] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:06.742662 10753 leader_election.cc:302] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dde49d6cb9714df1855843b9d8dfc90f; no voters:
I20260808 10:40:06.742959 10753 leader_election.cc:288] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:40:06.743156 10758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:06.743512 10758 raft_consensus.cc:686] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 LEADER]: Becoming Leader. State: Replica: dde49d6cb9714df1855843b9d8dfc90f, State: Running, Role: LEADER
I20260808 10:40:06.743847 10758 consensus_queue.cc:237] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:06.744280 10753 sys_catalog.cc:564] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:40:06.745877 10760 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dde49d6cb9714df1855843b9d8dfc90f. Latest consensus state: current_term: 1 leader_uuid: "dde49d6cb9714df1855843b9d8dfc90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } } }
I20260808 10:40:06.745878 10759 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dde49d6cb9714df1855843b9d8dfc90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } } }
I20260808 10:40:06.746096 10759 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:06.746086 10760 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:06.746564 10769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:06.749889 10769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:06.756228 10769 catalog_manager.cc:1347] Generated new cluster ID: b65039e8afa142599c8721577db70401
I20260808 10:40:06.756310 10769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:06.771513 10769 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:40:06.772650 10769 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:06.783152 10769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Generated new TSK 0
I20260808 10:40:06.784394 10769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:40:06.825626 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42553
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:06.937687 10781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:06.937997 10781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:06.938084 10781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:06.942205 10781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:06.942287 10781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:06.942375 10781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:40:06.947916 10781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:06.949008 10781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:06.950104 10781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:06.959669 10786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:06.959759 10787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:06.959921 10781 server_base.cc:1034] running on GCE node
W20260808 10:40:06.959786 10789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:06.960436 10781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:06.961052 10781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:06.962244 10781 hybrid_clock.cc:648] HybridClock initialized: now 1786185606962209 us; error 43 us; skew 500 ppm
Aug 08 10:40:06 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185606, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:06.965164 10781 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:40:06.966032 10781 webserver.cc:460] Webserver started at http://127.1.194.65:38605/ using document root <none> and password file <none>
I20260808 10:40:06.966611 10781 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:06.966677 10781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:06.966913 10781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:06.968894 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "bd7bec40facd4832b78e300e60967822"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "0028c13a68545eca8929a34d3f027ce9"
server_key_iv: "20332621fb9efd0fc356386b747ca466"
server_key_version: "encryptionkey@0"
I20260808 10:40:06.969463 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "bd7bec40facd4832b78e300e60967822"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "0028c13a68545eca8929a34d3f027ce9"
server_key_iv: "20332621fb9efd0fc356386b747ca466"
server_key_version: "encryptionkey@0"
I20260808 10:40:06.973145 10781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:06.975636 10797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:06.976818 10781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:06.976953 10781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "bd7bec40facd4832b78e300e60967822"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "0028c13a68545eca8929a34d3f027ce9"
server_key_iv: "20332621fb9efd0fc356386b747ca466"
server_key_version: "encryptionkey@0"
I20260808 10:40:06.977061 10781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:07.016762 10781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:07.020150 10781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:07.020352 10781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:07.020946 10781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:07.021806 10781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:07.021883 10781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.021952 10781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:07.021986 10781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.030617 10781 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:33249
I20260808 10:40:07.030646 10919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:33249 every 8 connection(s)
I20260808 10:40:07.031481 10781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:40:07.033170 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 10781
I20260808 10:40:07.033339 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:40:07.033751 1801 external_mini_cluster.cc:1428] Setting key 2a02eb10427e74e0a3038967152856c3
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185606, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:40:07.047273 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.033453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:44643 (local address 127.1.194.126:42553)
0808 10:40:07.033740 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:07.033745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:07.034511 (+ 766us) server_negotiation.cc:407] Connection header received
0808 10:40:07.035841 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:07.035848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:07.035913 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:07.036024 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:07.037250 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.037893 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.039532 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.039766 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.039923 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:07.043082 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:07.043085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:07.043087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:07.043114 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:07.044819 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:07.045551 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:07.045555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:07.045557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:07.045663 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:07.046079 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:07.046084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:07.046085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:07.046342 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:07.046444 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:07.046963 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 10:40:07.047102 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:40:07.048282 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.033889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:44643)
0808 10:40:07.034362 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:07.034415 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:07.035636 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:07.036170 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:07.036176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:07.036600 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:07.037078 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:07.037091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.038023 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.038027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:07.039315 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:07.039330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.040126 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.040132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:07.040377 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:07.041201 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:07.041219 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:07.042861 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:07.045027 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:07.045032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:07.045046 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:07.045369 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:07.045770 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:07.045774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:07.045776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:07.045989 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:07.046453 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:07.046462 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:07.046728 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 10:40:07.046922 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:40:07.047269 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":135,"threads_started":1}
I20260808 10:40:07.049913 10921 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42553
I20260808 10:40:07.050171 10921 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:07.050716 10921 heartbeater.cc:511] Master 127.1.194.126:42553 requested a full tablet report, sending...
I20260808 10:40:07.052279 10712 ts_manager.cc:194] Registered new tserver with Master: bd7bec40facd4832b78e300e60967822 (127.1.194.65:33249)
I20260808 10:40:07.053443 10712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:44643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:40:07.065614 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42553
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:07.176024 10926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:07.176275 10926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:07.176317 10926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:07.180318 10926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:07.180374 10926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:07.180459 10926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:40:07.185767 10926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:07.186806 10926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:07.187891 10926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:07.197216 10931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.197265 10934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:07.197283 10926 server_base.cc:1034] running on GCE node
W20260808 10:40:07.197221 10932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:07.197829 10926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:07.198454 10926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:07.199661 10926 hybrid_clock.cc:648] HybridClock initialized: now 1786185607199643 us; error 35 us; skew 500 ppm
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:07.208619 10926 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:40:07.209695 10926 webserver.cc:460] Webserver started at http://127.1.194.66:34407/ using document root <none> and password file <none>
I20260808 10:40:07.210300 10926 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:07.210366 10926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:07.210597 10926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:07.212718 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "22cf8f6654d649d594ec2aabf2033797"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "65af0e29a5572e13a0472c6a16f6bc0d"
server_key_iv: "9673c87289fd1d68b24385355d7196d0"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.213291 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "22cf8f6654d649d594ec2aabf2033797"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "65af0e29a5572e13a0472c6a16f6bc0d"
server_key_iv: "9673c87289fd1d68b24385355d7196d0"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.217123 10926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 10:40:07.219652 10941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.220822 10926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:07.220920 10926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "22cf8f6654d649d594ec2aabf2033797"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "65af0e29a5572e13a0472c6a16f6bc0d"
server_key_iv: "9673c87289fd1d68b24385355d7196d0"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.221004 10926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:07.245632 10926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:07.248963 10926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:07.249145 10926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:07.249825 10926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:07.250681 10926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:07.250732 10926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.250777 10926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:07.250800 10926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.260134 10926 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:40765
I20260808 10:40:07.260156 11066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:40765 every 8 connection(s)
I20260808 10:40:07.261080 10926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:40:07.262420 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 10926
I20260808 10:40:07.262576 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:40:07.262962 1801 external_mini_cluster.cc:1428] Setting key 4f8524038f7d04398a6d06403cdc9627
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:40:07.276140 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.263093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:58153 (local address 127.1.194.126:42553)
0808 10:40:07.263229 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:07.263232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:07.264201 (+ 969us) server_negotiation.cc:407] Connection header received
0808 10:40:07.265623 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:07.265626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:07.265681 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:07.265765 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:07.266889 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.267433 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.268603 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.268829 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.268940 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:07.271977 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:07.271980 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:07.271982 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:07.272006 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:07.273648 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:07.274297 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:07.274300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:07.274301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:07.274375 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:07.274741 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:07.274743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:07.274744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:07.275034 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:07.275146 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:07.275726 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 10:40:07.275858 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:40:07.276993 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.263571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:58153)
0808 10:40:07.264065 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:07.264108 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:07.265345 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:07.265931 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:07.265938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:07.266286 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:07.266751 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:07.266764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.267573 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.267576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:07.268449 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:07.268457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.268980 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.268984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:07.269158 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:07.269808 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:07.269825 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:07.271798 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:07.273766 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:07.273777 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:07.273799 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:07.274199 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:07.274488 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:07.274491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:07.274493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:07.274646 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:07.275177 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:07.275183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:07.275439 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 10:40:07.275630 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 10:40:07.275906 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":146,"threads_started":1}
I20260808 10:40:07.278318 11067 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42553
I20260808 10:40:07.278585 11067 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:07.279203 11067 heartbeater.cc:511] Master 127.1.194.126:42553 requested a full tablet report, sending...
I20260808 10:40:07.280349 10712 ts_manager.cc:194] Registered new tserver with Master: 22cf8f6654d649d594ec2aabf2033797 (127.1.194.66:40765)
I20260808 10:40:07.280932 10712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:58153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:40:07.293411 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42553
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:07.406042 11073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:07.406318 11073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:07.406385 11073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:07.410406 11073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:07.410480 11073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:07.410560 11073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:40:07.415686 11073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:07.416690 11073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:07.417716 11073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:07.427264 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.427229 11078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.427229 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:07.427346 11073 server_base.cc:1034] running on GCE node
I20260808 10:40:07.427908 11073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:07.428550 11073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:07.429692 11073 hybrid_clock.cc:648] HybridClock initialized: now 1786185607429677 us; error 26 us; skew 500 ppm
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:07.432407 11073 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:40:07.433373 11073 webserver.cc:460] Webserver started at http://127.1.194.67:38295/ using document root <none> and password file <none>
I20260808 10:40:07.434031 11073 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:07.434106 11073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:07.434283 11073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:07.436316 11073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "2cc09279b8bd48109e3a57b0b43c8665"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "c905c41749933b3bcc8fb6638c9fc4dd"
server_key_iv: "17c3bd7bebadfe18de230569c4a3b2e9"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.436880 11073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "2cc09279b8bd48109e3a57b0b43c8665"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "c905c41749933b3bcc8fb6638c9fc4dd"
server_key_iv: "17c3bd7bebadfe18de230569c4a3b2e9"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.441033 11073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:40:07.443665 11090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.445149 11073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:07.445287 11073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "2cc09279b8bd48109e3a57b0b43c8665"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "c905c41749933b3bcc8fb6638c9fc4dd"
server_key_iv: "17c3bd7bebadfe18de230569c4a3b2e9"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.445403 11073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:07.460384 11073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:07.463615 11073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:07.463830 11073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:07.464455 11073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:07.465337 11073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:07.465413 11073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.465538 11073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:07.465592 11073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.475353 11073 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:46431
I20260808 10:40:07.475421 11211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:46431 every 8 connection(s)
I20260808 10:40:07.476312 11073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:40:07.480279 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11073
I20260808 10:40:07.480502 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:40:07.480930 1801 external_mini_cluster.cc:1428] Setting key e32fee3d63b91111e6a59c49a6b5eef7
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:40:07.491495 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.478284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:51391 (local address 127.1.194.126:42553)
0808 10:40:07.478419 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:07.478423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:07.479306 (+ 883us) server_negotiation.cc:407] Connection header received
0808 10:40:07.480718 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:07.480722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:07.480778 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:07.480858 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:07.482347 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.482980 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.484291 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.484669 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.484791 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:07.487554 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:07.487558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:07.487561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:07.487589 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:07.489197 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:07.489843 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:07.489847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:07.489850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:07.489946 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:07.490328 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:07.490332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:07.490334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:07.490535 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:07.490650 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:07.491163 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 10:40:07.491295 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:40:07.492298 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.478718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:51391)
0808 10:40:07.479166 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:07.479208 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:07.480418 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:07.481035 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:07.481051 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:07.481557 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:07.482214 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:07.482230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.483102 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.483105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:07.484007 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:07.484017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:07.484831 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:07.484836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:07.485033 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:07.485725 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:07.485743 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:07.487377 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:07.489351 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:07.489357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:07.489372 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:07.489732 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:07.490054 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:07.490057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:07.490059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:07.490219 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:07.490646 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:07.490653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:07.490909 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 10:40:07.491101 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 10:40:07.491455 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":125,"threads_started":1}
I20260808 10:40:07.493606 11212 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42553
I20260808 10:40:07.493849 11212 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:07.494488 11212 heartbeater.cc:511] Master 127.1.194.126:42553 requested a full tablet report, sending...
I20260808 10:40:07.495609 10712 ts_manager.cc:194] Registered new tserver with Master: 2cc09279b8bd48109e3a57b0b43c8665 (127.1.194.67:46431)
I20260808 10:40:07.496166 10712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:51391
I20260808 10:40:07.507556 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab.
Entry for principal oryx/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab.
WARNING: no policy specified for oryx/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab.
Entry for principal oryx/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab.
WARNING: no policy specified for oryx/127.1.194.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab.
Entry for principal oryx/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab.
I20260808 10:40:07.590297 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 10781
I20260808 10:40:07.597096 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 10926
I20260808 10:40:07.603470 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11073
I20260808 10:40:07.609896 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 10677
I20260808 10:40:07.617044 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42553
--webserver_interface=127.1.194.126
--webserver_port=33057
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:07.727859 11222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:07.728138 11222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:07.728178 11222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:07.732215 11222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:07.732276 11222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:07.732295 11222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:07.732311 11222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:07.732362 11222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:07.737905 11222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:07.738970 11222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:07.739991 11222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:07.748762 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.748806 11228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.748762 11230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:07.749243 11222 server_base.cc:1034] running on GCE node
I20260808 10:40:07.749845 11222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:07.750933 11222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:07.752103 11222 hybrid_clock.cc:648] HybridClock initialized: now 1786185607752088 us; error 32 us; skew 500 ppm
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:07.755172 11222 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:40:07.756083 11222 webserver.cc:460] Webserver started at http://127.1.194.126:33057/ using document root <none> and password file <none>
I20260808 10:40:07.756662 11222 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:07.756769 11222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:07.760598 11222 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:07.762748 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.763891 11222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:40:07.764043 11222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "dde49d6cb9714df1855843b9d8dfc90f"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "35fcb8c877c478f28e68f2fa47377d2b"
server_key_iv: "f8d40280c74766beb6bde1dc275189c8"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.764591 11222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:07.774902 11222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:07.777683 11222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:07.777869 11222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:07.784739 11222 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:42553
I20260808 10:40:07.784766 11292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:42553 every 8 connection(s)
I20260808 10:40:07.785789 11222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:40:07.789882 11293 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:40:07.791489 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Bootstrap starting.
I20260808 10:40:07.794116 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11222
I20260808 10:40:07.794631 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:33249
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:42553
--webserver_port=38605
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:40:07.808835 11293 log.cc:824] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:07.815591 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:40:07.815981 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Bootstrap complete.
I20260808 10:40:07.820355 11293 raft_consensus.cc:348] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:07.820742 11293 raft_consensus.cc:729] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde49d6cb9714df1855843b9d8dfc90f, State: Initialized, Role: FOLLOWER
I20260808 10:40:07.821208 11293 consensus_queue.cc:260] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:07.821324 11293 raft_consensus.cc:388] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:40:07.821409 11293 raft_consensus.cc:482] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:40:07.821548 11293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:40:07.823422 11293 raft_consensus.cc:504] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:07.823750 11293 leader_election.cc:302] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dde49d6cb9714df1855843b9d8dfc90f; no voters:
I20260808 10:40:07.824091 11293 leader_election.cc:288] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:40:07.824183 11298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:40:07.824393 11298 raft_consensus.cc:686] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [term 2 LEADER]: Becoming Leader. State: Replica: dde49d6cb9714df1855843b9d8dfc90f, State: Running, Role: LEADER
I20260808 10:40:07.824689 11298 consensus_queue.cc:237] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } }
I20260808 10:40:07.825349 11293 sys_catalog.cc:564] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:40:07.825639 11300 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dde49d6cb9714df1855843b9d8dfc90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } } }
I20260808 10:40:07.826383 11300 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:07.826272 11299 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dde49d6cb9714df1855843b9d8dfc90f. Latest consensus state: current_term: 2 leader_uuid: "dde49d6cb9714df1855843b9d8dfc90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde49d6cb9714df1855843b9d8dfc90f" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42553 } } }
I20260808 10:40:07.826745 11299 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:07.827033 11308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:07.830284 11308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:07.831189 11308 catalog_manager.cc:1259] Loaded cluster ID: b65039e8afa142599c8721577db70401
I20260808 10:40:07.831254 11308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:07.833550 11308 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:07.834204 11308 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dde49d6cb9714df1855843b9d8dfc90f: Loaded TSK: 0
I20260808 10:40:07.835393 11308 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:40:07.910317 11296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:07.910571 11296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:07.910660 11296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:07.915529 11296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:07.915622 11296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:07.915738 11296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:40:07.923070 11296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.65.keytab
--principal=oryx/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:33249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=38605
--tserver_master_addrs=127.1.194.126:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:07.924100 11296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:07.925123 11296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:07.934532 11322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.934590 11323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:07.934829 11325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:07.935585 11296 server_base.cc:1034] running on GCE node
I20260808 10:40:07.936064 11296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:07.936834 11296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:07.938046 11296 hybrid_clock.cc:648] HybridClock initialized: now 1786185607938011 us; error 43 us; skew 500 ppm
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185607, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:07.941695 11296 init.cc:377] Logged in from keytab as oryx/127.1.194.65@KRBTEST.COM (short username oryx)
I20260808 10:40:07.942916 11296 webserver.cc:460] Webserver started at http://127.1.194.65:38605/ using document root <none> and password file <none>
I20260808 10:40:07.943658 11296 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:07.943778 11296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:07.948969 11296 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:07.951571 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.953104 11296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:07.953225 11296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "bd7bec40facd4832b78e300e60967822"
format_stamp: "Formatted at 2026-08-08 10:40:06 on dist-test-slave-ss4n"
server_key: "0028c13a68545eca8929a34d3f027ce9"
server_key_iv: "20332621fb9efd0fc356386b747ca466"
server_key_version: "encryptionkey@0"
I20260808 10:40:07.953886 11296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:07.979736 11296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:07.982828 11296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:07.983044 11296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:07.983639 11296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:07.984453 11296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:07.984522 11296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.984601 11296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:07.984639 11296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:07.993216 11296 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:33249
I20260808 10:40:07.993275 11459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:33249 every 8 connection(s)
I20260808 10:40:07.994305 11296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:40:08.002251 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11296
I20260808 10:40:08.002810 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:40765
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:42553
--webserver_port=34407
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:40:07 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.007809 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:07.996477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:38111)
0808 10:40:07.997436 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:07.997515 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:07.998317 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:07.999337 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:07.999343 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:07.999680 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.000123 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.000137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.001379 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.001382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.002395 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.002407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.003429 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.003432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.003565 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.004182 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.004197 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.006840 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":600,"threads_started":1}
W20260808 10:40:08.007802 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:07.996378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:38111 (local address 127.1.194.126:42553)
0808 10:40:07.996857 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:07.996873 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:07.997657 (+ 784us) server_negotiation.cc:407] Connection header received
0808 10:40:07.998699 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:07.998710 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:07.998990 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:07.999251 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.000295 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.001263 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.002525 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.003274 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.003477 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.007058 (+ 3581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:38111: BlockingRecv error: recv got EOF from 127.1.194.65:38111 (error 108)
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":145,"threads_started":1}
W20260808 10:40:08.008538 11460 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:42553 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.194.126:42553: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.014678 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.008972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:45639)
0808 10:40:08.009094 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.009109 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.009183 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.009463 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.009465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.009640 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.009732 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.009737 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.010841 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.010844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.011374 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.011381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.012024 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.012026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.012105 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.012638 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.012649 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.014583 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:40:08.014899 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.009082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:45639 (local address 127.1.194.126:42553)
0808 10:40:08.009218 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.009225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.009243 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:40:08.009288 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.009293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.009351 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.009457 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.009847 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.010727 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.011487 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.011911 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.012039 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.014736 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:45639: BlockingRecv error: recv got EOF from 127.1.194.65:45639 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.020242 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.015258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:46257)
0808 10:40:08.015453 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.015466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.015524 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.015751 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.015753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.015867 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.015929 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.015933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.016887 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.016890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.017376 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.017382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.017870 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.017873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.017950 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.018412 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.018422 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.020110 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260808 10:40:08.020329 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.015376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:46257 (local address 127.1.194.126:42553)
0808 10:40:08.015497 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.015501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.015515 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:08.015601 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.015604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.015647 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.015720 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.016019 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.016792 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.017556 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.017757 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.017859 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.020218 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:46257: BlockingRecv error: recv got EOF from 127.1.194.65:46257 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:40:08.020438 11460 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:40:08.115902 11464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:08.116158 11464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:08.116199 11464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:08.120163 11464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:08.120218 11464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:08.120314 11464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:40:08.125368 11464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.66.keytab
--principal=oryx/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:40765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=34407
--tserver_master_addrs=127.1.194.126:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:08.126399 11464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:08.127341 11464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:08.136468 11474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:08.136482 11472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:08.136478 11470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:08.136847 11464 server_base.cc:1034] running on GCE node
I20260808 10:40:08.137219 11464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:08.137898 11464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:08.139084 11464 hybrid_clock.cc:648] HybridClock initialized: now 1786185608139053 us; error 50 us; skew 500 ppm
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185608, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:08.141806 11464 init.cc:377] Logged in from keytab as oryx/127.1.194.66@KRBTEST.COM (short username oryx)
I20260808 10:40:08.142663 11464 webserver.cc:460] Webserver started at http://127.1.194.66:34407/ using document root <none> and password file <none>
I20260808 10:40:08.143187 11464 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:08.143260 11464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:08.146978 11464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:40:08.148975 11481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:08.150233 11464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:08.150326 11464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "22cf8f6654d649d594ec2aabf2033797"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "65af0e29a5572e13a0472c6a16f6bc0d"
server_key_iv: "9673c87289fd1d68b24385355d7196d0"
server_key_version: "encryptionkey@0"
I20260808 10:40:08.150815 11464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:08.175350 11464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:08.178615 11464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:08.178771 11464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:08.179297 11464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:08.180058 11464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:08.180104 11464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:08.180143 11464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:08.180164 11464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:08.189328 11464 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:40765
I20260808 10:40:08.189414 11607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:40765 every 8 connection(s)
I20260808 10:40:08.190541 11464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.200489 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11464
I20260808 10:40:08.200984 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:46431
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:42553
--webserver_port=38295
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:08.201097 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.192268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:32989 (local address 127.1.194.126:42553)
0808 10:40:08.192409 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.192413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.193228 (+ 815us) server_negotiation.cc:407] Connection header received
0808 10:40:08.194544 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.194548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.194601 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.194679 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.195761 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.196272 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.197366 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.197582 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.197651 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.200967 (+ 3316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:32989: BlockingRecv error: recv got EOF from 127.1.194.66:32989 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:40:08.201862 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.192649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:32989)
0808 10:40:08.193076 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.193120 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.194356 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.194828 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.194834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.195166 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.195643 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.195656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.196372 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.196374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.197269 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.197278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.197705 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.197708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.197839 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.198431 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.198446 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.200854 (+ 2408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
W20260808 10:40:08.202771 11608 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:42553 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.194.126:42553: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.208830 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.203261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:55227)
0808 10:40:08.203381 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.203396 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.203477 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.203700 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.203703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.203860 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.203946 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.203951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.204653 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.204659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.205631 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.205647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.206106 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.206113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.206245 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.206814 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.206826 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.208644 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:40:08.208843 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.203327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:55227 (local address 127.1.194.126:42553)
0808 10:40:08.203443 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.203448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.203461 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:40:08.203560 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.203564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.203606 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.203685 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.204071 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.204548 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.205762 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.205986 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.206076 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.208753 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:55227: BlockingRecv error: recv got EOF from 127.1.194.66:55227 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.215412 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.209411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:53371)
0808 10:40:08.209589 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.209608 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.209697 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.209987 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.209992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.210176 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.210293 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.210303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.211098 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.211103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.212073 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.212087 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.212528 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.212535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.212655 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.213160 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.213181 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.215279 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:40:08.215591 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.209540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:53371 (local address 127.1.194.126:42553)
0808 10:40:08.209641 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.209644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.209656 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:40:08.209817 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.209819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.209859 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.209935 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.210428 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.210983 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.212214 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.212418 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.212500 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.215414 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:53371: BlockingRecv error: recv got EOF from 127.1.194.66:53371 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 10:40:08.215792 11608 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:40:08.316459 11611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:08.316745 11611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:08.316790 11611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:08.321015 11611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:08.321089 11611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:08.321177 11611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:40:08.326856 11611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/krb5kdc/oryx_127.1.194.67.keytab
--principal=oryx/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:46431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=38295
--tserver_master_addrs=127.1.194.126:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:08.328037 11611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:08.329052 11611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:08.339074 11620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:08.339090 11616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:08.339154 11617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:08.339362 11611 server_base.cc:1034] running on GCE node
I20260808 10:40:08.339848 11611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:08.340451 11611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:08.341621 11611 hybrid_clock.cc:648] HybridClock initialized: now 1786185608341590 us; error 43 us; skew 500 ppm
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185608, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:08.344551 11611 init.cc:377] Logged in from keytab as oryx/127.1.194.67@KRBTEST.COM (short username oryx)
I20260808 10:40:08.345703 11611 webserver.cc:460] Webserver started at http://127.1.194.67:38295/ using document root <none> and password file <none>
I20260808 10:40:08.346287 11611 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:08.346402 11611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:08.350317 11611 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 10:40:08.352356 11627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:08.353714 11611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:40:08.353845 11611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "2cc09279b8bd48109e3a57b0b43c8665"
format_stamp: "Formatted at 2026-08-08 10:40:07 on dist-test-slave-ss4n"
server_key: "c905c41749933b3bcc8fb6638c9fc4dd"
server_key_iv: "17c3bd7bebadfe18de230569c4a3b2e9"
server_key_version: "encryptionkey@0"
I20260808 10:40:08.354418 11611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:08.382853 11611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:08.386245 11611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:08.386458 11611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:08.387099 11611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:08.387990 11611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:08.388075 11611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:08.388154 11611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:08.388196 11611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:08.398274 11611 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:46431
I20260808 10:40:08.398321 11753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:46431 every 8 connection(s)
I20260808 10:40:08.399343 11611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:40:08.400179 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11611
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.411223 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.401298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.1.194.126:42553)
0808 10:40:08.401433 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.401438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.401454 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:40:08.401560 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.401566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.401641 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.401776 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.402786 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.403674 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.404980 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.405187 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.405264 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.406927 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:08.406932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:08.406945 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:08.406973 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:08.408827 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:08.409281 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:08.409285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:08.409289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:08.409401 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:08.409760 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:08.409763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:08.409764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:08.410190 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:08.410405 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:08.410714 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 10:40:08.411096 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 10:40:08.411770 11757 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.401778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:56227 (local address 127.1.194.126:42553)
0808 10:40:08.402037 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.402042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.403630 (+ 1588us) server_negotiation.cc:407] Connection header received
0808 10:40:08.404265 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.404267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.404306 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.404366 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.405905 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.406429 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.407750 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.408094 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.408210 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.411651 (+ 3441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:56227: BlockingRecv error: recv got EOF from 127.1.194.67:56227 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
I20260808 10:40:08.412263 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.401758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:56227)
0808 10:40:08.402344 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.402409 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.404051 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.404537 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.404546 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.405065 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.405753 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.405771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.406569 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.406572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.407628 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.407638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.408233 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.408237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.408385 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.408960 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.408976 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.411382 (+ 2406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":145,"threads_started":1}
W20260808 10:40:08.412801 11754 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:42553 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.194.126:42553: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.418859 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.413254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:40669)
0808 10:40:08.413552 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.413567 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.413655 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.413891 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.413894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.414091 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.414189 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.414195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.415065 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.415069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.415736 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.415745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.416220 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.416224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.416325 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.416855 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.416875 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.418710 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:40:08.418948 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.413300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:40669 (local address 127.1.194.126:42553)
0808 10:40:08.413399 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.413401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.413630 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:40:08.413739 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.413742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.413783 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.413876 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.414305 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.414942 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.415855 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.416107 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.416218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.418827 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:40669: BlockingRecv error: recv got EOF from 127.1.194.67:40669 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:08 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:08.424192 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:08.419491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:35859)
0808 10:40:08.419594 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:08.419610 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:08.419692 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:08.419920 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:08.419922 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:08.420062 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:08.420131 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.420136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.420798 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.420800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.421373 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:08.421381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.421916 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.421919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:08.421979 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.422461 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:08.422474 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:08.424104 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:40:08.424358 11462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:08.419600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:35859 (local address 127.1.194.126:42553)
0808 10:40:08.419745 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:08.419748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:08.419759 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:40:08.419793 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:08.419795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:08.419828 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:08.419892 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:08.420229 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.420709 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.421591 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:08.421806 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:08.421889 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:08.424264 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:35859: BlockingRecv error: recv got EOF from 127.1.194.67:35859 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 10:40:08.424575 11754 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 10:40:09 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:09 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:09.026512 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:09.021084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:43191)
0808 10:40:09.021341 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:09.021356 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:09.021452 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:09.021730 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:09.021732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:09.021907 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:09.022001 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:09.022006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.022757 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.022760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:09.023329 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:09.023338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.023788 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.023790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:09.023850 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:09.024389 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:09.024401 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:09.026394 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
W20260808 10:40:09.026723 11761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:09.021121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:43191 (local address 127.1.194.126:42553)
0808 10:40:09.021361 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:09.021364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:09.021406 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:40:09.021571 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:09.021573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:09.021615 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:09.021692 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:09.022114 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.022663 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.023433 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.023691 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.023773 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:09.026555 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:43191: BlockingRecv error: recv got EOF from 127.1.194.65:43191 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
Aug 08 10:40:09 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:09 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:09.222255 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:09.216548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:36227)
0808 10:40:09.216828 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:09.216841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:09.216942 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:09.217213 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:09.217216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:09.217398 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:09.217528 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:09.217534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.218560 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.218564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:09.219287 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:09.219298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.219727 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.219730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:09.219789 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:09.220269 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:09.220280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:09.222126 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
W20260808 10:40:09.222385 11761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:09.216654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:36227 (local address 127.1.194.126:42553)
0808 10:40:09.216821 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:09.216824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:09.216913 (+ 89us) server_negotiation.cc:407] Connection header received
0808 10:40:09.217024 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:09.217045 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:09.217092 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:09.217170 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:09.217697 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.218435 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.219391 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.219622 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.219712 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:09.222267 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:36227: BlockingRecv error: recv got EOF from 127.1.194.66:36227 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 10:40:09 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:09 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:09.430907 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:09.425218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:35691)
0808 10:40:09.425586 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:09.425598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:09.425706 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:09.426007 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:09.426010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:09.426190 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:09.426287 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:09.426292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.427046 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.427048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:09.427605 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:09.427613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.428146 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.428148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:09.428208 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:09.428759 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:09.428771 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:09.430730 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":82,"threads_started":1}
W20260808 10:40:09.430966 11761 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:09.425285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:35691 (local address 127.1.194.126:42553)
0808 10:40:09.425449 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:09.425453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:09.425653 (+ 200us) server_negotiation.cc:407] Connection header received
0808 10:40:09.425806 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:09.425808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:09.425853 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:09.425921 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:09.426410 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.426951 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.427775 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:09.428026 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:09.428120 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:09.430854 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:35691: BlockingRecv error: recv got EOF from 127.1.194.67:35691 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:40:10 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:10 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:10.032910 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:10.027306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:42843)
0808 10:40:10.027585 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:10.027599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:10.027701 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:10.028119 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:10.028122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:10.028300 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:10.028398 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:10.028403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.029239 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.029242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:10.029854 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:10.029863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.030327 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.030330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:10.030391 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:10.030917 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:10.030928 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:10.032773 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
W20260808 10:40:10.033077 11765 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:10.027346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:42843 (local address 127.1.194.126:42553)
0808 10:40:10.027586 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:10.027590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:10.027667 (+ 77us) server_negotiation.cc:407] Connection header received
0808 10:40:10.027859 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:10.027864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:10.027963 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:10.028089 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:10.028583 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.029134 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.029969 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.030206 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.030292 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:10.032907 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:42843: BlockingRecv error: recv got EOF from 127.1.194.65:42843 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":65,"threads_started":1}
Aug 08 10:40:10 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:10 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:10.229164 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:10.223070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:52625)
0808 10:40:10.223415 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:10.223432 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:10.223572 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:10.223887 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:10.223890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:10.224118 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:10.224213 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:10.224219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.225297 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.225300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:10.226056 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:10.226071 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.226583 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.226586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:10.226647 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:10.227205 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:10.227216 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:10.229029 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":98,"threads_started":1}
W20260808 10:40:10.229328 11765 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:10.223207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:52625 (local address 127.1.194.126:42553)
0808 10:40:10.223414 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:10.223421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:10.223539 (+ 118us) server_negotiation.cc:407] Connection header received
0808 10:40:10.223710 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:10.223713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:10.223773 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:10.223870 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:10.224337 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.225174 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.226168 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.226398 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.226588 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:10.229165 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:52625: BlockingRecv error: recv got EOF from 127.1.194.66:52625 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 10:40:10 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:10 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:10.437476 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:10.431765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:57497)
0808 10:40:10.432037 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:10.432051 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:10.432152 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:10.432396 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:10.432398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:10.432584 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:10.432684 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:10.432689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.433408 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.433410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:10.434018 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:10.434026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.434494 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.434496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:10.434554 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:10.435048 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:10.435061 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:10.437286 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
W20260808 10:40:10.437541 11765 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:10.431928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:57497 (local address 127.1.194.126:42553)
0808 10:40:10.432091 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:10.432095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:10.432110 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:10.432236 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:10.432238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:10.432285 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:10.432357 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:10.432780 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.433317 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.434137 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:10.434356 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:10.434447 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:10.437410 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:57497: BlockingRecv error: recv got EOF from 127.1.194.67:57497 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 10:40:11 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:11 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:11.040786 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:11.033726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:50827)
0808 10:40:11.033990 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:11.034011 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:11.034113 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:11.034359 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:11.034361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:11.034667 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:11.034798 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:11.034807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.035787 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.035794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:11.036801 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:11.036817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.037304 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.037308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:11.037392 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:11.038077 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:11.038091 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:11.040614 (+ 2523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
W20260808 10:40:11.040913 11769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:11.033822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:50827 (local address 127.1.194.126:42553)
0808 10:40:11.034076 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:11.034079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:11.034094 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:11.034200 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:11.034202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:11.034246 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:11.034327 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:11.034920 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.035676 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.036949 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.037187 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.037298 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:11.040765 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:50827: BlockingRecv error: recv got EOF from 127.1.194.65:50827 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":109,"threads_started":1}
Aug 08 10:40:11 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:11 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:11.235671 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:11.230028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:60665)
0808 10:40:11.230309 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:11.230323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:11.230457 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:11.230740 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:11.230743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:11.230986 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:11.231114 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:11.231122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.231939 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.231945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:11.232504 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:11.232512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.232962 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.232966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:11.233047 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:11.233630 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:11.233645 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:11.235493 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":72,"threads_started":1}
W20260808 10:40:11.235711 11769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:11.230081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:60665 (local address 127.1.194.126:42553)
0808 10:40:11.230239 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:11.230244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:11.230400 (+ 156us) server_negotiation.cc:407] Connection header received
0808 10:40:11.230557 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:11.230560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:11.230620 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:11.230717 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:11.231236 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.231836 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.232632 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.232851 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.232958 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:11.235613 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:60665: BlockingRecv error: recv got EOF from 127.1.194.66:60665 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 10:40:11 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:11 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:11.443941 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:11.438402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:43283)
0808 10:40:11.438791 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:11.438805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:11.438909 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:11.439187 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:11.439190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:11.439371 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:11.439469 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:11.439474 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.440226 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.440228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:11.440778 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:11.440787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.441226 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.441229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:11.441292 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:11.441788 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:11.441799 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:11.443770 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":167,"threads_started":1}
W20260808 10:40:11.444001 11769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:11.438450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:43283 (local address 127.1.194.126:42553)
0808 10:40:11.438649 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:11.438652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:11.438905 (+ 253us) server_negotiation.cc:407] Connection header received
0808 10:40:11.438978 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:11.438980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:11.439045 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:11.439117 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:11.439589 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.440126 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.440892 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:11.441126 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:11.441207 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:11.443888 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:43283: BlockingRecv error: recv got EOF from 127.1.194.67:43283 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 08 10:40:12 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:12 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:12.048148 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:12.042058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:34893)
0808 10:40:12.042390 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:12.042405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:12.042511 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:12.042803 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:12.042806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:12.042989 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:12.043111 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:12.043117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.044047 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.044049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:12.044790 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:12.044801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.045308 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.045311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:12.045374 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:12.045999 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:12.046011 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:12.048030 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":109,"threads_started":1}
W20260808 10:40:12.048395 11773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:12.042187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:34893 (local address 127.1.194.126:42553)
0808 10:40:12.042527 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:12.042532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:12.042548 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:40:12.042596 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:12.042599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:12.042647 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:12.042738 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:12.043232 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.043925 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.044928 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.045196 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.045294 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:12.048223 (+ 2929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:34893: BlockingRecv error: recv got EOF from 127.1.194.65:34893 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
Aug 08 10:40:12 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:12 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:12.245611 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:12.236576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:55629)
0808 10:40:12.236867 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:12.236880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:12.236982 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:12.237210 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:12.237212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:12.237393 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:12.237534 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:12.237542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.238266 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.238268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:12.238813 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:12.238820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.239229 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.239232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:12.239290 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:12.243488 (+ 4198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:12.243500 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:12.245445 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
W20260808 10:40:12.245788 11773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:12.236631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:55629 (local address 127.1.194.126:42553)
0808 10:40:12.236779 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:12.236783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:12.236946 (+ 163us) server_negotiation.cc:407] Connection header received
0808 10:40:12.237047 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:12.237049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:12.237097 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:12.237164 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:12.237639 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.238172 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.238915 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.239134 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.239214 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:12.245660 (+ 6446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:55629: BlockingRecv error: recv got EOF from 127.1.194.66:55629 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 10:40:12 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:12 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:12.450312 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:12.444815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:40441)
0808 10:40:12.445116 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:12.445132 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:12.445255 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:12.445586 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:12.445589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:12.445772 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:12.445869 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:12.445874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.446612 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.446614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:12.447177 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:12.447185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.447612 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.447615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:12.447676 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:12.448264 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:12.448276 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:12.450198 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
W20260808 10:40:12.450464 11773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:12.444994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:40441 (local address 127.1.194.126:42553)
0808 10:40:12.445133 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:12.445136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:12.445205 (+ 69us) server_negotiation.cc:407] Connection header received
0808 10:40:12.445338 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:12.445340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:12.445388 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:12.445454 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:12.445970 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.446484 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.447282 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:12.447506 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:12.447602 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:12.450334 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:40441: BlockingRecv error: recv got EOF from 127.1.194.67:40441 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 10:40:13 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:13 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:13.055164 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:13.049054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:39033)
0808 10:40:13.049354 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:13.049368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:13.049460 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:13.049798 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:13.049803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:13.050018 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:13.050117 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:13.050123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.050969 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.050973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:13.051542 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:13.051550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.052135 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.052139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:13.052239 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:13.052848 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:13.052861 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:13.055017 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":118,"threads_started":1}
W20260808 10:40:13.055300 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:13.049178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:39033 (local address 127.1.194.126:42553)
0808 10:40:13.049432 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:13.049436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:13.049452 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:40:13.049591 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:13.049593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:13.049640 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:13.049769 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:13.050232 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.050854 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.051664 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.051910 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.052003 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:13.055135 (+ 3132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:39033: BlockingRecv error: recv got EOF from 127.1.194.65:39033 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":93,"threads_started":1}
Aug 08 10:40:13 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:13 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:13.252710 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:13.246587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:41077)
0808 10:40:13.246875 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:13.246891 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:13.246986 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:13.247245 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:13.247247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:13.247517 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:13.247621 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:13.247627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.248670 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.248674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:13.249362 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:13.249373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.249882 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.249885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:13.249948 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:13.250477 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:13.250496 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:13.252594 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
W20260808 10:40:13.252837 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:13.246766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:41077 (local address 127.1.194.126:42553)
0808 10:40:13.246929 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:13.246932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:13.246947 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:13.247085 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:13.247087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:13.247129 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:13.247203 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:13.247754 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.248540 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.249498 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.249766 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.249853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:13.252724 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:41077: BlockingRecv error: recv got EOF from 127.1.194.66:41077 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 10:40:13 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:13 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:13.457734 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:13.451417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:38217)
0808 10:40:13.451734 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:13.451757 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:13.451910 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:13.452227 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:13.452230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:13.452472 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:13.452571 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:13.452577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.453372 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.453378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:13.454149 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:13.454159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.454696 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.454700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:13.454786 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:13.455351 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:13.455363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:13.457526 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":116,"threads_started":1}
W20260808 10:40:13.457795 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:13.451464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:38217 (local address 127.1.194.126:42553)
0808 10:40:13.451694 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:13.451699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:13.451848 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:40:13.452049 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:13.452052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:13.452110 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:13.452200 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:13.452693 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.453255 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.454275 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:13.454578 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:13.454691 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:13.457683 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:38217: BlockingRecv error: recv got EOF from 127.1.194.67:38217 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 08 10:40:14 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:14 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:14.061767 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:14.056048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:50643)
0808 10:40:14.056322 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:14.056336 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:14.056442 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:14.056706 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:14.056708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:14.056889 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:14.056982 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:14.056987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.057946 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.057949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:14.058619 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:14.058628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.059070 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.059073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:14.059132 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:14.059757 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:14.059769 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:14.061651 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":94,"threads_started":1}
W20260808 10:40:14.061967 11782 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:14.056098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:50643 (local address 127.1.194.126:42553)
0808 10:40:14.056337 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:14.056340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:14.056410 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:40:14.056522 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:14.056524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:14.056575 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:14.056660 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:14.057137 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.057823 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.058741 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.058960 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.059040 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:14.061791 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:50643: BlockingRecv error: recv got EOF from 127.1.194.65:50643 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
Aug 08 10:40:14 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:14 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:14.259841 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:14.253662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:46463)
0808 10:40:14.253935 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:14.253951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:14.254052 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:14.254303 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:14.254306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:14.254497 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:14.254601 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:14.254606 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.255412 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.255415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:14.256000 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:14.256008 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.256565 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.256569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:14.256652 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:14.257229 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:14.257243 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:14.259723 (+ 2480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
W20260808 10:40:14.259943 11782 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:14.253712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:46463 (local address 127.1.194.126:42553)
0808 10:40:14.253874 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:14.253879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:14.254020 (+ 141us) server_negotiation.cc:407] Connection header received
0808 10:40:14.254119 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:14.254121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:14.254167 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:14.254235 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:14.254708 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.255303 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.256121 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.256338 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.256419 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:14.259853 (+ 3434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:46463: BlockingRecv error: recv got EOF from 127.1.194.66:46463 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 10:40:14 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:14 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:14.464955 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:14.458654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:43133)
0808 10:40:14.458935 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:14.458951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:14.459064 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:14.459463 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:14.459466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:14.459677 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:14.459781 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:14.459787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.460707 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.460711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:14.461331 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:14.461340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.461900 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.461903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:14.461970 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:14.462614 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:14.462627 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:14.464786 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":109,"threads_started":1}
W20260808 10:40:14.465013 11782 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:14.458863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:43133 (local address 127.1.194.126:42553)
0808 10:40:14.459039 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:14.459042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:14.459061 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:40:14.459142 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:14.459144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:14.459268 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:14.459385 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:14.459922 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.460583 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.461539 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:14.461792 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:14.461891 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:14.464907 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:43133: BlockingRecv error: recv got EOF from 127.1.194.67:43133 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 10:40:15 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:15 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:15.068398 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:15.062708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:51103)
0808 10:40:15.063084 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:15.063098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:15.063202 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:15.063448 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:15.063450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:15.063644 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:15.063741 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:15.063747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.064488 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.064491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:15.065053 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:15.065062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.065635 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.065639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:15.065728 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:15.066303 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:15.066320 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:15.068285 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":76,"threads_started":1}
W20260808 10:40:15.068764 11786 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:15.062779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:51103 (local address 127.1.194.126:42553)
0808 10:40:15.063091 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:15.063095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:15.063170 (+ 75us) server_negotiation.cc:407] Connection header received
0808 10:40:15.063284 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:15.063286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:15.063332 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:15.063413 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:15.063846 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.064388 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.065182 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.065475 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.065633 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:15.068627 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:51103: BlockingRecv error: recv got EOF from 127.1.194.65:51103 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":137,"threads_started":1}
Aug 08 10:40:15 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:15 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:15.267433 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:15.260877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:58415)
0808 10:40:15.261298 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:15.261315 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:15.261471 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:15.261862 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:15.261865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:15.262088 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:15.262197 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:15.262202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.262997 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.263002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:15.263569 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:15.263578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.264154 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.264159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:15.264255 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:15.264904 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:15.264947 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:15.267318 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":96,"threads_started":1}
W20260808 10:40:15.267620 11786 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:15.261060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:58415 (local address 127.1.194.126:42553)
0808 10:40:15.261271 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:15.261276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:15.261407 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:40:15.261678 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:15.261682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:15.261739 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:15.261833 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:15.262322 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.262874 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.263696 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.263975 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.264096 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:15.267478 (+ 3382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:58415: BlockingRecv error: recv got EOF from 127.1.194.66:58415 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 08 10:40:15 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:15 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:15.472151 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:15.465925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:54789)
0808 10:40:15.466299 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:15.466313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:15.466425 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:15.466677 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:15.466679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:15.466864 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:15.466958 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:15.466963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.467817 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.467821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:15.468691 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:15.468702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.469228 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.469235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:15.469328 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:15.469899 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:15.469912 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:15.472020 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":127,"threads_started":1}
W20260808 10:40:15.472371 11786 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:15.466000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:54789 (local address 127.1.194.126:42553)
0808 10:40:15.466196 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:15.466201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:15.466395 (+ 194us) server_negotiation.cc:407] Connection header received
0808 10:40:15.466507 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:15.466509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:15.466558 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:15.466651 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:15.467056 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.467688 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.468815 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:15.469105 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:15.469186 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:15.472230 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:54789: BlockingRecv error: recv got EOF from 127.1.194.67:54789 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 08 10:40:16 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:16 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:16.074988 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:16.069156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:53863)
0808 10:40:16.069452 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:16.069467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:16.069623 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:16.069880 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:16.069883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:16.070075 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:16.070176 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:16.070182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.070946 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.070949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:16.071572 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:16.071581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.072055 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.072058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:16.072128 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:16.072740 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:16.072753 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:16.074805 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
W20260808 10:40:16.075101 11790 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:16.069299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:53863 (local address 127.1.194.126:42553)
0808 10:40:16.069586 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:16.069590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:16.069607 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:40:16.069713 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:16.069715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:16.069763 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:16.069848 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:16.070276 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.070845 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.071682 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.071923 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.072009 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:16.074933 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:53863: BlockingRecv error: recv got EOF from 127.1.194.65:53863 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
Aug 08 10:40:16 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:16 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:16.274811 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:16.268246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:36787)
0808 10:40:16.268556 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:16.268575 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:16.268798 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:16.269160 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:16.269163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:16.269397 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:16.269594 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:16.269603 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.270628 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.270634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:16.271269 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:16.271278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.271783 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.271787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:16.271883 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:16.272507 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:16.272521 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:16.274625 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":91,"threads_started":1}
W20260808 10:40:16.274873 11790 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:16.268303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:36787 (local address 127.1.194.126:42553)
0808 10:40:16.268586 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:16.268592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:16.268719 (+ 127us) server_negotiation.cc:407] Connection header received
0808 10:40:16.268936 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:16.268941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:16.269027 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:16.269129 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:16.269734 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.270504 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.271406 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.271657 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.271778 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:16.274755 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:36787: BlockingRecv error: recv got EOF from 127.1.194.66:36787 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 08 10:40:16 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:16 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:16.479173 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:16.472980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:52475)
0808 10:40:16.473280 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:16.473294 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:16.473395 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:16.473721 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:16.473723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:16.473912 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:16.474008 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:16.474014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.474795 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.474797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:16.475352 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:16.475360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.475811 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.475814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:16.475877 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:16.476542 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:16.476555 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:16.479044 (+ 2489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
W20260808 10:40:16.479362 11790 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:16.473090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:52475 (local address 127.1.194.126:42553)
0808 10:40:16.473315 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:16.473319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:16.473339 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:40:16.473516 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:16.473520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:16.473576 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:16.473655 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:16.474161 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.474688 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.475469 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:16.475703 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:16.475789 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:16.479206 (+ 3417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:52475: BlockingRecv error: recv got EOF from 127.1.194.67:52475 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 08 10:40:17 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:17 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:17.081777 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:17.075929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:47293)
0808 10:40:17.076212 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:17.076225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:17.076315 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:17.076620 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:17.076622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:17.076808 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:17.076928 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:17.076934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.077803 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.077806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:17.078368 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:17.078376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.078841 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.078843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:17.078902 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:17.079432 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:17.079443 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:17.081606 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
W20260808 10:40:17.081975 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:17.076004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:47293 (local address 127.1.194.126:42553)
0808 10:40:17.076291 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:17.076294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:17.076311 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:40:17.076409 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:17.076412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:17.076460 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:17.076603 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:17.077041 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.077696 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.078469 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.078743 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.078829 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:17.081788 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:47293: BlockingRecv error: recv got EOF from 127.1.194.65:47293 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":128,"threads_started":1}
Aug 08 10:40:17 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:17 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:17.284992 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:17.275778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:45127)
0808 10:40:17.276071 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:17.276085 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:17.276185 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:17.276481 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:17.276483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:17.276705 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:17.276801 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:17.276806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.277619 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.277622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:17.278175 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:17.278183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.278648 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.278650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:17.278710 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:17.282706 (+ 3996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:17.282719 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:17.284860 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
W20260808 10:40:17.285202 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:17.275865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:45127 (local address 127.1.194.126:42553)
0808 10:40:17.276024 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:17.276028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:17.276165 (+ 137us) server_negotiation.cc:407] Connection header received
0808 10:40:17.276306 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:17.276309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:17.276359 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:17.276436 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:17.276911 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.277430 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.278305 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.278546 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.278629 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:17.285024 (+ 6395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:45127: BlockingRecv error: recv got EOF from 127.1.194.66:45127 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 10:40:17 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:17 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:17.486267 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:17.480120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:40761)
0808 10:40:17.480456 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:17.480476 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:17.480625 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:17.480903 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:17.480906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:17.481122 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:17.481214 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:17.481220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.482096 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.482101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:17.482671 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:17.482680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.483212 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.483216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:17.483302 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:17.483877 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:17.483899 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:17.486137 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
W20260808 10:40:17.486410 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:17.480253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:40761 (local address 127.1.194.126:42553)
0808 10:40:17.480413 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:17.480417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:17.480554 (+ 137us) server_negotiation.cc:407] Connection header received
0808 10:40:17.480728 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:17.480731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:17.480786 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:17.480873 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:17.481337 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.481958 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.482797 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:17.483099 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:17.483208 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:17.486282 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:40761: BlockingRecv error: recv got EOF from 127.1.194.67:40761 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 10:40:18 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:18 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:18.088608 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:18.082698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:38817)
0808 10:40:18.083012 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:18.083026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:18.083113 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:18.083368 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:18.083371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:18.083577 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:18.083676 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:18.083682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.084448 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.084451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:18.085082 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:18.085095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.085731 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.085738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:18.085832 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:18.086481 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:18.086500 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:18.088395 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
W20260808 10:40:18.088649 11798 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:18.082812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:38817 (local address 127.1.194.126:42553)
0808 10:40:18.083071 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:18.083074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:18.083088 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:18.083205 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:18.083207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:18.083249 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:18.083330 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:18.083783 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.084340 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.085242 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.085596 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.085712 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:18.088526 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:38817: BlockingRecv error: recv got EOF from 127.1.194.65:38817 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
Aug 08 10:40:18 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:18 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:18.291828 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:18.285975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:38215)
0808 10:40:18.286338 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:18.286360 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:18.286480 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:18.286783 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:18.286785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:18.286964 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:18.287063 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:18.287068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.287971 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.287973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:18.288522 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:18.288528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.288988 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.288990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:18.289061 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:18.289662 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:18.289677 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:18.291712 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":135,"threads_started":1}
W20260808 10:40:18.292033 11798 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:18.286034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:38215 (local address 127.1.194.126:42553)
0808 10:40:18.286204 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:18.286209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:18.286450 (+ 241us) server_negotiation.cc:407] Connection header received
0808 10:40:18.286582 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:18.286585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:18.286649 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:18.286735 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:18.287200 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.287866 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.288654 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.288878 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.288975 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:18.291887 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:38215: BlockingRecv error: recv got EOF from 127.1.194.66:38215 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 10:40:18 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:18 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:18.493069 11800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:18.487222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:55857)
0808 10:40:18.487590 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:18.487604 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:18.487702 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:18.487930 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:18.487933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:18.488129 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:18.488229 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:18.488235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.489001 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.489005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:18.489693 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:18.489704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.490128 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.490131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:18.490199 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:18.490706 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:18.490719 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:18.492945 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":131,"threads_started":1}
W20260808 10:40:18.493242 11798 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:18.487353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:55857 (local address 127.1.194.126:42553)
0808 10:40:18.487494 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:18.487497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:18.487678 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:40:18.487772 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:18.487774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:18.487821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:18.487887 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:18.488338 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.488869 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.489806 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:18.490016 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:18.490105 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:18.493107 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:55857: BlockingRecv error: recv got EOF from 127.1.194.67:55857 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 10:40:19 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:19 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:19.095223 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:19.089559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:42107)
0808 10:40:19.089880 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:19.089898 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:19.090017 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:19.090333 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:19.090336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:19.090554 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:19.090658 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:19.090664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.091441 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.091447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:19.092003 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:19.092010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.092494 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.092497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:19.092583 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:19.093119 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:19.093130 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:19.095029 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
W20260808 10:40:19.095288 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:19.089603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:42107 (local address 127.1.194.126:42553)
0808 10:40:19.089880 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:19.089886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:19.089960 (+ 74us) server_negotiation.cc:407] Connection header received
0808 10:40:19.090148 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:19.090151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:19.090208 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:19.090305 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:19.090789 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.091329 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.092134 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.092371 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.092477 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:19.095166 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:42107: BlockingRecv error: recv got EOF from 127.1.194.65:42107 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
Aug 08 10:40:19 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:19 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:19.299659 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:19.293100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:34105)
0808 10:40:19.293397 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:19.293411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:19.293582 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:19.293886 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:19.293888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:19.294100 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:19.294208 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:19.294214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.295191 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.295194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:19.295818 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:19.295827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.296348 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.296352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:19.296493 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:19.297144 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:19.297159 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:19.299483 (+ 2324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
W20260808 10:40:19.299731 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:19.293244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:34105 (local address 127.1.194.126:42553)
0808 10:40:19.293429 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:19.293434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:19.293470 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:40:19.293699 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:19.293705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:19.293768 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:19.293845 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:19.294360 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.295037 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.295960 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.296198 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.296290 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:19.299606 (+ 3316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:34105: BlockingRecv error: recv got EOF from 127.1.194.66:34105 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 10:40:19 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:19 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:19.500844 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:19.494136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:57297)
0808 10:40:19.494639 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:19.494655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:19.494817 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:19.495172 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:19.495176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:19.495465 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:19.495619 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:19.495627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.496513 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.496517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:19.497161 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:19.497171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.497786 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.497791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:19.497886 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:19.498584 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:19.498598 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:19.500719 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":105,"threads_started":1}
W20260808 10:40:19.501081 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:19.494232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:57297 (local address 127.1.194.126:42553)
0808 10:40:19.494410 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:19.494413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:19.494760 (+ 347us) server_negotiation.cc:407] Connection header received
0808 10:40:19.494956 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:19.494961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:19.495029 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:19.495141 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:19.495767 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.496359 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.497301 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:19.497651 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:19.497785 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:19.500951 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:57297: BlockingRecv error: recv got EOF from 127.1.194.67:57297 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 10:40:20 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:20 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:20.102439 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:20.096303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:44627)
0808 10:40:20.096608 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:20.096622 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:20.096747 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:20.097035 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:20.097038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:20.097281 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:20.097402 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:20.097411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.098575 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.098578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:20.099139 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:20.099148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.099615 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.099617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:20.099681 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:20.100192 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:20.100203 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:20.102269 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
W20260808 10:40:20.102535 11807 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:20.096364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:44627 (local address 127.1.194.126:42553)
0808 10:40:20.096615 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:20.096619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:20.096694 (+ 75us) server_negotiation.cc:407] Connection header received
0808 10:40:20.096843 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:20.096845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:20.096890 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:20.096978 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:20.097643 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.098432 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.099251 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.099493 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.099582 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:20.102403 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:44627: BlockingRecv error: recv got EOF from 127.1.194.65:44627 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
Aug 08 10:40:20 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:20 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:20.306074 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:20.300589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:35425)
0808 10:40:20.300866 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:20.300879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:20.300980 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:20.301210 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:20.301212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:20.301394 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:20.301518 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:20.301524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.302254 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.302256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:20.302837 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:20.302846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.303289 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.303292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:20.303355 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:20.303858 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:20.303870 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:20.305935 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
W20260808 10:40:20.306272 11807 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:20.300728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:35425 (local address 127.1.194.126:42553)
0808 10:40:20.300899 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:20.300902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:20.300931 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:40:20.301060 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:20.301062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:20.301104 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:20.301173 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:20.301605 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.302122 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.302949 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.303180 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.303260 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:20.306091 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:35425: BlockingRecv error: recv got EOF from 127.1.194.66:35425 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 10:40:20 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:20 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:20.507619 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:20.501663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:46927)
0808 10:40:20.501996 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:20.502009 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:20.502107 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:20.502418 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:20.502421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:20.502694 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:20.502797 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:20.502803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.503675 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.503678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:20.504228 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:20.504236 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.504908 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.504911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:20.504980 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:20.505645 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:20.505658 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:20.507494 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":106,"threads_started":1}
W20260808 10:40:20.507823 11807 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:20.501788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:46927 (local address 127.1.194.126:42553)
0808 10:40:20.501956 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:20.501960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:20.502085 (+ 125us) server_negotiation.cc:407] Connection header received
0808 10:40:20.502205 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:20.502208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:20.502262 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:20.502349 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:20.502939 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.503494 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.504369 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:20.504774 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:20.504881 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:20.507682 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:46927: BlockingRecv error: recv got EOF from 127.1.194.67:46927 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 10:40:21 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:21 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:21.109364 11810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:21.103287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:42185)
0808 10:40:21.103656 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:21.103673 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:21.103804 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:21.104171 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:21.104174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:21.104409 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:21.104511 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:21.104518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.105366 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.105371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:21.106026 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:21.106035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.106490 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.106494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:21.106584 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:21.107202 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:21.107214 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:21.109185 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
W20260808 10:40:21.109440 11811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:21.103322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:42185 (local address 127.1.194.126:42553)
0808 10:40:21.103629 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:21.103633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:21.103777 (+ 144us) server_negotiation.cc:407] Connection header received
0808 10:40:21.103900 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:21.103903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:21.103989 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:21.104139 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:21.104640 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.105255 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.106145 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.106377 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.106485 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:21.109313 (+ 2828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:42185: BlockingRecv error: recv got EOF from 127.1.194.65:42185 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
Aug 08 10:40:21 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:21 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:21.312830 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:21.306924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:48905)
0808 10:40:21.307219 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:21.307233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:21.307328 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:21.307622 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:21.307624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:21.307850 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:21.307944 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:21.307950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.308710 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.308712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:21.309315 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:21.309323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.309826 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.309829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:21.309895 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:21.310465 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:21.310477 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:21.312699 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":120,"threads_started":1}
W20260808 10:40:21.313021 11811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:21.307072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:48905 (local address 127.1.194.126:42553)
0808 10:40:21.307242 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:21.307246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:21.307302 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:40:21.307411 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:21.307414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:21.307458 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:21.307549 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:21.308059 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.308605 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.309436 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.309719 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.309804 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:21.312850 (+ 3046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:48905: BlockingRecv error: recv got EOF from 127.1.194.66:48905 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 10:40:21 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:21 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:21.514496 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:21.508369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:38201)
0808 10:40:21.508677 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:21.508691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:21.508795 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:21.509062 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:21.509065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:21.509252 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:21.509348 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:21.509353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.510210 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.510214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:21.510962 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:21.510975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.511513 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.511516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:21.511625 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:21.512170 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:21.512181 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:21.514357 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
W20260808 10:40:21.514670 11811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:21.508446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:38201 (local address 127.1.194.126:42553)
0808 10:40:21.508600 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:21.508607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:21.508789 (+ 182us) server_negotiation.cc:407] Connection header received
0808 10:40:21.508885 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:21.508887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:21.508957 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:21.509029 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:21.509469 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.510075 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.511098 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:21.511350 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:21.511454 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:21.514508 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:38201: BlockingRecv error: recv got EOF from 127.1.194.67:38201 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 10:40:22 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:22 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:22.116389 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:22.110252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:55611)
0808 10:40:22.110664 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:22.110678 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:22.110774 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:22.111081 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:22.111084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:22.111306 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:22.111401 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:22.111407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.112196 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.112199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:22.112799 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:22.112809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.113345 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.113349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:22.113438 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:22.114224 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:22.114240 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:22.116264 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
W20260808 10:40:22.116605 11815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:22.110483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:55611 (local address 127.1.194.126:42553)
0808 10:40:22.110818 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:22.110821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:22.110835 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:40:22.110873 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:22.110875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:22.110916 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:22.110998 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:22.111537 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.112073 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.112936 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.113182 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.113290 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:22.116405 (+ 3115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:55611: BlockingRecv error: recv got EOF from 127.1.194.65:55611 (error 108)
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
Aug 08 10:40:22 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:22 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:22.327370 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:22.313737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:43901)
0808 10:40:22.314057 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:22.314075 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:22.314242 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:22.314584 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:22.314587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:22.314847 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:22.314955 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:22.314961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.315825 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.315832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:22.316500 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:22.316510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.317061 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.317065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:22.317170 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:22.324951 (+ 7781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:22.324966 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:22.327221 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":116,"threads_started":1}
W20260808 10:40:22.327505 11815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:22.313828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:43901 (local address 127.1.194.126:42553)
0808 10:40:22.314001 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:22.314006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:22.314185 (+ 179us) server_negotiation.cc:407] Connection header received
0808 10:40:22.314370 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:22.314376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:22.314436 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:22.314564 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:22.315095 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.315694 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.316644 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.316927 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.317057 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:22.327374 (+ 10317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:43901: BlockingRecv error: recv got EOF from 127.1.194.66:43901 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 10:40:22 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:22 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.67@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:22.521363 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:22.515566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.67:36861)
0808 10:40:22.515942 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:22.515955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:22.516122 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:22.516398 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:22.516401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:22.516614 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:22.516712 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:22.516718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.517516 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.517521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:22.518102 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:22.518111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.518610 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.518614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:22.518708 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:22.519270 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:22.519282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:22.521179 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":126,"threads_started":1}
W20260808 10:40:22.521423 11815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:22.515605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:36861 (local address 127.1.194.126:42553)
0808 10:40:22.515773 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:22.515777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:22.516063 (+ 286us) server_negotiation.cc:407] Connection header received
0808 10:40:22.516226 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:22.516230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:22.516284 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:22.516369 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:22.516836 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.517368 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.518230 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:22.518480 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:22.518594 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:22.521308 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.67:36861: BlockingRecv error: recv got EOF from 127.1.194.67:36861 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 10:40:23 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:23 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.65@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:23.123849 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:23.117428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.65:35029)
0808 10:40:23.117817 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:23.117832 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:23.117939 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:23.118209 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:23.118211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:23.118409 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:23.118564 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:23.118570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.119355 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.119358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:23.120338 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:23.120353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.120968 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.120975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:23.121097 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.121775 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:23.121809 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:23.123728 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":175,"threads_started":1}
W20260808 10:40:23.124034 11819 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:23.117607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:35029 (local address 127.1.194.126:42553)
0808 10:40:23.117882 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:23.117886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:23.117901 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:40:23.118034 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:23.118036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:23.118089 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:23.118168 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:23.118669 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.119212 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.120504 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.120803 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.120954 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.123879 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.65:35029: BlockingRecv error: recv got EOF from 127.1.194.65:35029 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
Aug 08 10:40:23 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:40:23 dist-test-slave-ss4n krb5kdc[10664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.194.66@KRBTEST.COM for oryx/127.1.194.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:40:23.334326 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:23.328414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42553 (local address 127.1.194.66:43611)
0808 10:40:23.328744 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:23.328758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:23.328856 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:23.329112 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:23.329114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:23.329302 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:23.329397 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:23.329402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.330195 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.330198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:23.330926 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:23.330939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.331558 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.331564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:23.331659 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.332282 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:23.332293 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:23.334213 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.194.126:42553: Server oryx/127.1.194.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":154,"threads_started":1}
W20260808 10:40:23.334497 11819 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:23.328543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:43611 (local address 127.1.194.126:42553)
0808 10:40:23.328719 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:23.328723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:23.328879 (+ 156us) server_negotiation.cc:407] Connection header received
0808 10:40:23.328934 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:23.328936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:23.328985 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:23.329077 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:23.329521 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.330084 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.331111 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.331429 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.331522 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.334342 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.194.66:43611: BlockingRecv error: recv got EOF from 127.1.194.66:43611 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:40:23.402086 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11296
I20260808 10:40:23.409713 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11464
I20260808 10:40:23.415843 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11611
I20260808 10:40:23.422058 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11222
2026-08-08T10:40:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18126 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T10:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:23Z Disabled control of system clock
I20260808 10:40:23.455842 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:32925
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:32925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:40:23.579638 11825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:23.580003 11825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:23.580089 11825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:23.584687 11825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:23.584787 11825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:23.584820 11825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:23.584865 11825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:23.584905 11825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:23.590161 11825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:32925
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:23.591311 11825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:23.592903 11825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:40:23.602432 11825 server_base.cc:1034] running on GCE node
W20260808 10:40:23.602438 11832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:23.602388 11835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:23.602685 11833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:23.603255 11825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:23.604388 11825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:23.605619 11825 hybrid_clock.cc:648] HybridClock initialized: now 1786185623605604 us; error 29 us; skew 500 ppm
I20260808 10:40:23.607638 11825 webserver.cc:460] Webserver started at http://127.1.194.126:35547/ using document root <none> and password file <none>
I20260808 10:40:23.608248 11825 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:23.608340 11825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:23.608566 11825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:23.610741 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "aecf4d6024ab43f08747c654faac6e8d"
format_stamp: "Formatted at 2026-08-08 10:40:23 on dist-test-slave-ss4n"
server_key: "58f3550fbf6b49716266b04f7c4a054e"
server_key_iv: "d8f5303dd5ce8f9de588f61b0dfd159e"
server_key_version: "encryptionkey@0"
I20260808 10:40:23.611323 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "aecf4d6024ab43f08747c654faac6e8d"
format_stamp: "Formatted at 2026-08-08 10:40:23 on dist-test-slave-ss4n"
server_key: "58f3550fbf6b49716266b04f7c4a054e"
server_key_iv: "d8f5303dd5ce8f9de588f61b0dfd159e"
server_key_version: "encryptionkey@0"
I20260808 10:40:23.615193 11825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 10:40:23.617736 11841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:23.619156 11825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:40:23.619341 11825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "aecf4d6024ab43f08747c654faac6e8d"
format_stamp: "Formatted at 2026-08-08 10:40:23 on dist-test-slave-ss4n"
server_key: "58f3550fbf6b49716266b04f7c4a054e"
server_key_iv: "d8f5303dd5ce8f9de588f61b0dfd159e"
server_key_version: "encryptionkey@0"
I20260808 10:40:23.619498 11825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:23.653443 11825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:23.654294 11825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:23.654508 11825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:23.661856 11825 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:32925
I20260808 10:40:23.661880 11898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:32925 every 8 connection(s)
I20260808 10:40:23.662854 11825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:40:23.665989 11899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:23.671502 11899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d: Bootstrap starting.
I20260808 10:40:23.673177 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11825
I20260808 10:40:23.673326 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:40:23.673775 1801 external_mini_cluster.cc:1428] Setting key 72d97f259541635b484c9a6556602f64
I20260808 10:40:23.674311 11899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:23.675220 11899 log.cc:824] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:23.677356 11899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d: No bootstrap required, opened a new log
I20260808 10:40:23.680836 11899 raft_consensus.cc:348] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecf4d6024ab43f08747c654faac6e8d" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 32925 } }
I20260808 10:40:23.681099 11899 raft_consensus.cc:374] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:23.681175 11899 raft_consensus.cc:729] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aecf4d6024ab43f08747c654faac6e8d, State: Initialized, Role: FOLLOWER
I20260808 10:40:23.681706 11899 consensus_queue.cc:260] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecf4d6024ab43f08747c654faac6e8d" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 32925 } }
I20260808 10:40:23.681893 11899 raft_consensus.cc:388] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:40:23.681984 11899 raft_consensus.cc:482] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:40:23.682124 11899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:23.683128 11899 raft_consensus.cc:504] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecf4d6024ab43f08747c654faac6e8d" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 32925 } }
I20260808 10:40:23.683491 11899 leader_election.cc:302] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aecf4d6024ab43f08747c654faac6e8d; no voters:
I20260808 10:40:23.683847 11899 leader_election.cc:288] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:40:23.683928 11907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:23.684150 11907 raft_consensus.cc:686] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [term 1 LEADER]: Becoming Leader. State: Replica: aecf4d6024ab43f08747c654faac6e8d, State: Running, Role: LEADER
I20260808 10:40:23.684315 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:23.675017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.1.194.126:32925)
0808 10:40:23.676002 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:23.676028 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:23.676077 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:40:23.676870 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:23.676889 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:23.677240 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:23.677731 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:23.678196 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.679262 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.680292 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.680822 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.681021 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.682002 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:23.682009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:23.682022 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:23.682566 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:23.682627 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:23.682650 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:23.682771 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:23.682950 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 10:40:23.683262 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":147,"threads_started":1}
I20260808 10:40:23.684504 11907 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecf4d6024ab43f08747c654faac6e8d" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 32925 } }
I20260808 10:40:23.685158 11899 sys_catalog.cc:564] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:40:23.686949 11908 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aecf4d6024ab43f08747c654faac6e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecf4d6024ab43f08747c654faac6e8d" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 32925 } } }
I20260808 10:40:23.687071 11908 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:23.687427 11909 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecf4d6024ab43f08747c654faac6e8d. Latest consensus state: current_term: 1 leader_uuid: "aecf4d6024ab43f08747c654faac6e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecf4d6024ab43f08747c654faac6e8d" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 32925 } } }
I20260808 10:40:23.687613 11909 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:23.687816 11917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:23.691301 11917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:23.696437 11917 catalog_manager.cc:1347] Generated new cluster ID: 330e8ec174864d0097a83ef87b546565
I20260808 10:40:23.696516 11917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:23.726481 11917 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:40:23.727623 11917 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:23.732157 11917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aecf4d6024ab43f08747c654faac6e8d: Generated new TSK 0
I20260808 10:40:23.732822 11917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:40:23.738586 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:32925
--builtin_ntp_servers=127.1.194.84:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 10:40:23.852910 11926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:23.853179 11926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:23.853215 11926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:23.857180 11926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:23.857237 11926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:23.857297 11926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:40:23.862028 11926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:32925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:23.863103 11926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:23.864460 11926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:23.874222 11932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:23.874233 11931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:23.874212 11934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:23.875139 11926 server_base.cc:1034] running on GCE node
I20260808 10:40:23.875527 11926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:23.876185 11926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:23.877348 11926 hybrid_clock.cc:648] HybridClock initialized: now 1786185623877329 us; error 30 us; skew 500 ppm
I20260808 10:40:23.879204 11926 webserver.cc:460] Webserver started at http://127.1.194.65:44245/ using document root <none> and password file <none>
I20260808 10:40:23.879765 11926 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:23.879824 11926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:23.880137 11926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:23.882213 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "3c446bdf1f344ddeb4a4c1f301017da1"
format_stamp: "Formatted at 2026-08-08 10:40:23 on dist-test-slave-ss4n"
server_key: "6a59515cdee2875e588cb58d96a021b8"
server_key_iv: "8a88ca6272be1143a4f1109df9ced648"
server_key_version: "encryptionkey@0"
I20260808 10:40:23.882771 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "3c446bdf1f344ddeb4a4c1f301017da1"
format_stamp: "Formatted at 2026-08-08 10:40:23 on dist-test-slave-ss4n"
server_key: "6a59515cdee2875e588cb58d96a021b8"
server_key_iv: "8a88ca6272be1143a4f1109df9ced648"
server_key_version: "encryptionkey@0"
I20260808 10:40:23.886302 11926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:23.888906 11940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:23.890261 11926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:40:23.890376 11926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "3c446bdf1f344ddeb4a4c1f301017da1"
format_stamp: "Formatted at 2026-08-08 10:40:23 on dist-test-slave-ss4n"
server_key: "6a59515cdee2875e588cb58d96a021b8"
server_key_iv: "8a88ca6272be1143a4f1109df9ced648"
server_key_version: "encryptionkey@0"
I20260808 10:40:23.890481 11926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:23.902673 11926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:23.903347 11926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:23.903542 11926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:23.904134 11926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:23.905071 11926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:23.905120 11926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:23.905186 11926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:23.905218 11926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:23.914482 11926 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:35151
I20260808 10:40:23.914513 12063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:35151 every 8 connection(s)
I20260808 10:40:23.915419 11926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:40:23.915727 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 11926
I20260808 10:40:23.915889 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:40:23.916254 1801 external_mini_cluster.cc:1428] Setting key 40737b76f4c8ad7472a69fa7bc8a0b92
I20260808 10:40:23.920063 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:32925
--builtin_ntp_servers=127.1.194.84:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:40:23.926816 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:23.917663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:58641 (local address 127.1.194.126:32925)
0808 10:40:23.917824 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:23.917829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:23.919210 (+ 1381us) server_negotiation.cc:407] Connection header received
0808 10:40:23.920941 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:23.920944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:23.921007 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:23.921086 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:23.922296 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.923164 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.924492 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.924779 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.924873 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.926312 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:23.926316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:23.926320 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:23.926337 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:23.926365 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:23.926374 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:23.926434 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:23.926650 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 10:40:23.926702 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:40:23.928057 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:23.918199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:32925 (local address 127.1.194.65:58641)
0808 10:40:23.919051 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:23.919105 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:23.920722 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:23.921264 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:23.921272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:23.921376 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:23.922128 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:23.922153 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.923332 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.923336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:23.924368 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:23.924378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:23.924941 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:23.924948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:23.925181 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:23.925989 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:40:23.925991 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:23.926050 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:40:23.926132 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:23.926466 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:23.926478 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:23.926486 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:40:23.926628 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 10:40:23.926947 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260808 10:40:23.929450 12064 heartbeater.cc:347] Connected to a master server at 127.1.194.126:32925
I20260808 10:40:23.929760 12064 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:23.930365 12064 heartbeater.cc:511] Master 127.1.194.126:32925 requested a full tablet report, sending...
I20260808 10:40:23.932411 11859 ts_manager.cc:194] Registered new tserver with Master: 3c446bdf1f344ddeb4a4c1f301017da1 (127.1.194.65:35151)
I20260808 10:40:23.934219 11859 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.65:58641
W20260808 10:40:24.035562 12067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:24.035830 12067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:24.035867 12067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:24.039796 12067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:24.039853 12067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:24.039907 12067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:40:24.044536 12067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:32925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:24.045614 12067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:24.047133 12067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:24.056509 12075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.056520 12073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.056627 12072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:24.057055 12067 server_base.cc:1034] running on GCE node
I20260808 10:40:24.057439 12067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:24.058122 12067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:24.059315 12067 hybrid_clock.cc:648] HybridClock initialized: now 1786185624059293 us; error 34 us; skew 500 ppm
I20260808 10:40:24.061339 12067 webserver.cc:460] Webserver started at http://127.1.194.66:38197/ using document root <none> and password file <none>
I20260808 10:40:24.061978 12067 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:24.062076 12067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:24.062318 12067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:24.064371 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "133d2b0e4bc74580b95feef1da38bacd"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "52af8c020470edef498d830224ddc18d"
server_key_iv: "a5c51792f0a8da02699e4650d89521af"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.064949 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "133d2b0e4bc74580b95feef1da38bacd"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "52af8c020470edef498d830224ddc18d"
server_key_iv: "a5c51792f0a8da02699e4650d89521af"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.068661 12067 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:40:24.071390 12081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.072724 12067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:24.072878 12067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "133d2b0e4bc74580b95feef1da38bacd"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "52af8c020470edef498d830224ddc18d"
server_key_iv: "a5c51792f0a8da02699e4650d89521af"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.073055 12067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:24.112419 12067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:24.113168 12067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:24.113376 12067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:24.114044 12067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:24.115025 12067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:24.115097 12067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.115173 12067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:24.115216 12067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.125070 12067 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:34853
I20260808 10:40:24.125111 12205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:34853 every 8 connection(s)
I20260808 10:40:24.126021 12067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:40:24.129820 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12067
I20260808 10:40:24.129961 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:40:24.130251 1801 external_mini_cluster.cc:1428] Setting key 7885a6282e5ac7c563a7a9280ef7eba7
I20260808 10:40:24.132381 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:32925
--builtin_ntp_servers=127.1.194.84:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:40:24.135489 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.127849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:42785 (local address 127.1.194.126:32925)
0808 10:40:24.127986 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:24.127989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:24.129001 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 10:40:24.130612 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:24.130618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:24.130684 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:24.130761 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:24.131674 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.132201 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.133447 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.133730 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.133845 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:24.134983 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:24.134987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:24.134990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:24.135003 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:24.135035 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:24.135042 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:24.135128 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:24.135289 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:40:24.135352 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:40:24.136304 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.128192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:32925 (local address 127.1.194.66:42785)
0808 10:40:24.128824 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:24.128863 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:24.130396 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:24.130960 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:24.130966 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:24.131054 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:24.131520 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:24.131532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.132331 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.132334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:24.133307 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:24.133316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.133884 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.133887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:24.134060 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:24.134699 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:40:24.134700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:24.134746 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:40:24.134811 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:24.135125 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:24.135135 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:24.135144 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:40:24.135276 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 10:40:24.135544 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
I20260808 10:40:24.137678 12206 heartbeater.cc:347] Connected to a master server at 127.1.194.126:32925
I20260808 10:40:24.137928 12206 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:24.138586 12206 heartbeater.cc:511] Master 127.1.194.126:32925 requested a full tablet report, sending...
I20260808 10:40:24.139750 11859 ts_manager.cc:194] Registered new tserver with Master: 133d2b0e4bc74580b95feef1da38bacd (127.1.194.66:34853)
I20260808 10:40:24.140635 11859 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.66:42785
W20260808 10:40:24.247023 12209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:24.247272 12209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:24.247308 12209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:24.251372 12209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:24.251431 12209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:24.251482 12209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:40:24.256124 12209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:32925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:24.257107 12209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:24.258405 12209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:24.267851 12215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.267858 12218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.267935 12214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:24.268244 12209 server_base.cc:1034] running on GCE node
I20260808 10:40:24.268718 12209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:24.269304 12209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:24.270576 12209 hybrid_clock.cc:648] HybridClock initialized: now 1786185624270557 us; error 31 us; skew 500 ppm
I20260808 10:40:24.273042 12209 webserver.cc:460] Webserver started at http://127.1.194.67:34477/ using document root <none> and password file <none>
I20260808 10:40:24.273705 12209 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:24.273775 12209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:24.273967 12209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:24.276357 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "6da3455432cb4a0ca2493ee2f42d8d50"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "f0a6dcf0f6805e4a7e3bb68da9f82e67"
server_key_iv: "76f4b99415d758e20f3a44d0fc8854ad"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.276973 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "6da3455432cb4a0ca2493ee2f42d8d50"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "f0a6dcf0f6805e4a7e3bb68da9f82e67"
server_key_iv: "76f4b99415d758e20f3a44d0fc8854ad"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.280984 12209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 10:40:24.283656 12225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.284924 12209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:24.285096 12209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "6da3455432cb4a0ca2493ee2f42d8d50"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "f0a6dcf0f6805e4a7e3bb68da9f82e67"
server_key_iv: "76f4b99415d758e20f3a44d0fc8854ad"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.285223 12209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:24.305069 12209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:24.305918 12209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:24.306145 12209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:24.306787 12209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:24.307811 12209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:24.307890 12209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.307976 12209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:24.308024 12209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.317463 12209 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:33305
I20260808 10:40:24.317529 12345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:33305 every 8 connection(s)
I20260808 10:40:24.318475 12209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:40:24.320483 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12209
I20260808 10:40:24.320681 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:40:24.320990 1801 external_mini_cluster.cc:1428] Setting key da8cf6dadcaa746054119ca783d2044d
I20260808 10:40:24.329255 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.320695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:46215 (local address 127.1.194.126:32925)
0808 10:40:24.320860 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:24.320866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:24.322252 (+ 1386us) server_negotiation.cc:407] Connection header received
0808 10:40:24.323948 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:24.323951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:24.324018 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:24.324094 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:24.325178 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.325848 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.327139 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.327390 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.327490 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:24.328780 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:24.328783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:24.328785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:24.328806 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:24.328829 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:24.328838 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:24.328914 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:24.329071 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:40:24.329129 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:40:24.330250 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.321195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:32925 (local address 127.1.194.67:46215)
0808 10:40:24.322049 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:24.322107 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:24.323729 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:24.324286 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:24.324293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:24.324420 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:24.325028 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:24.325045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.325979 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.325982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:24.326974 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:24.326984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:24.327541 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:24.327544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:24.327742 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:24.328496 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:40:24.328498 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:24.328542 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:40:24.328619 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:24.328933 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:24.328940 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:24.328944 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:40:24.329048 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:40:24.329286 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":196,"threads_started":1}
I20260808 10:40:24.331483 12346 heartbeater.cc:347] Connected to a master server at 127.1.194.126:32925
I20260808 10:40:24.331771 12346 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:24.332345 12346 heartbeater.cc:511] Master 127.1.194.126:32925 requested a full tablet report, sending...
I20260808 10:40:24.333467 11859 ts_manager.cc:194] Registered new tserver with Master: 6da3455432cb4a0ca2493ee2f42d8d50 (127.1.194.67:33305)
I20260808 10:40:24.334213 11859 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.67:46215
I20260808 10:40:24.335466 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:40:24.338810 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:24.337690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.1.194.126:32925)
0808 10:40:24.337844 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:24.337851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:24.338037 (+ 186us) server_negotiation.cc:407] Connection header received
0808 10:40:24.338212 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:24.338215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:24.338267 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:24.338354 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:24.338681 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48474: BlockingRecv error: recv got EOF from 127.0.0.1:48474 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:40:24.338956 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11926
I20260808 10:40:24.350358 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12067
I20260808 10:40:24.359447 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12209
I20260808 10:40:24.368324 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 11825
2026-08-08T10:40:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (944 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T10:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:24Z Disabled control of system clock
I20260808 10:40:24.399554 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:35761
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:35761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:40:24.510187 12358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:24.510479 12358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:24.510519 12358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:24.514716 12358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:24.514783 12358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:24.514799 12358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:24.514817 12358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:24.514855 12358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:24.519853 12358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:35761
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:35761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:24.520987 12358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:24.522430 12358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:24.531270 12363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.531328 12364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:24.531268 12358 server_base.cc:1034] running on GCE node
W20260808 10:40:24.531281 12366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:24.532058 12358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:24.533257 12358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:24.534423 12358 hybrid_clock.cc:648] HybridClock initialized: now 1786185624534395 us; error 41 us; skew 500 ppm
I20260808 10:40:24.536240 12358 webserver.cc:460] Webserver started at http://127.1.194.126:42039/ using document root <none> and password file <none>
I20260808 10:40:24.536803 12358 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:24.536864 12358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:24.537081 12358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:24.539183 12358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "2000f4c609a84a2a86a681cfdc38a871"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "1cf653af9edb89ba9b5fafa192690418"
server_key_iv: "9d1467772520002aa442710e917be752"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.539769 12358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "2000f4c609a84a2a86a681cfdc38a871"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "1cf653af9edb89ba9b5fafa192690418"
server_key_iv: "9d1467772520002aa442710e917be752"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.543715 12358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 10:40:24.546653 12373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.547931 12358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:24.548121 12358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "2000f4c609a84a2a86a681cfdc38a871"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "1cf653af9edb89ba9b5fafa192690418"
server_key_iv: "9d1467772520002aa442710e917be752"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.548255 12358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:24.553014 12358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:24.553675 12358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:24.553860 12358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:24.560636 12358 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:35761
I20260808 10:40:24.560655 12430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:35761 every 8 connection(s)
I20260808 10:40:24.561618 12358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:40:24.564844 12431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:24.565516 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12358
I20260808 10:40:24.565655 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:40:24.565991 1801 external_mini_cluster.cc:1428] Setting key 36dc7985b4f1a390b175858bb8432e32
I20260808 10:40:24.571431 12431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871: Bootstrap starting.
I20260808 10:40:24.571813 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.567481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.1.194.126:35761)
0808 10:40:24.568187 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:24.568201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:24.568244 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:40:24.568900 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:24.568918 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:24.569262 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:24.569695 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:24.570523 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:24.570528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:24.570544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:24.570555 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:24.570608 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:24.570624 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:24.570777 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:24.570846 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 10:40:24.571082 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":141,"threads_started":1}
I20260808 10:40:24.573886 12431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:24.574700 12431 log.cc:824] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:24.576700 12431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871: No bootstrap required, opened a new log
I20260808 10:40:24.580310 12431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2000f4c609a84a2a86a681cfdc38a871" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35761 } }
I20260808 10:40:24.580538 12431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:24.580613 12431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2000f4c609a84a2a86a681cfdc38a871, State: Initialized, Role: FOLLOWER
I20260808 10:40:24.581058 12431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2000f4c609a84a2a86a681cfdc38a871" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35761 } }
I20260808 10:40:24.581202 12431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:40:24.581245 12431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:40:24.581346 12431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:24.582484 12431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2000f4c609a84a2a86a681cfdc38a871" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35761 } }
I20260808 10:40:24.582835 12431 leader_election.cc:302] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2000f4c609a84a2a86a681cfdc38a871; no voters:
I20260808 10:40:24.583184 12431 leader_election.cc:288] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:40:24.583263 12436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:24.583451 12436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [term 1 LEADER]: Becoming Leader. State: Replica: 2000f4c609a84a2a86a681cfdc38a871, State: Running, Role: LEADER
I20260808 10:40:24.583738 12436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2000f4c609a84a2a86a681cfdc38a871" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35761 } }
I20260808 10:40:24.584489 12431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:40:24.585577 12438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2000f4c609a84a2a86a681cfdc38a871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2000f4c609a84a2a86a681cfdc38a871" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35761 } } }
I20260808 10:40:24.585700 12438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:24.585968 12437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2000f4c609a84a2a86a681cfdc38a871. Latest consensus state: current_term: 1 leader_uuid: "2000f4c609a84a2a86a681cfdc38a871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2000f4c609a84a2a86a681cfdc38a871" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35761 } } }
I20260808 10:40:24.586061 12437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871 [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:24.586104 12446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:24.589253 12446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:24.595101 12446 catalog_manager.cc:1347] Generated new cluster ID: 2a33a0981fc546088191de7b202b1d03
I20260808 10:40:24.595184 12446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:24.614923 12446 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:40:24.615265 12446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:24.623718 12446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2000f4c609a84a2a86a681cfdc38a871: Generated new TSK 0
I20260808 10:40:24.624476 12446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:40:24.635589 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:35761
--builtin_ntp_servers=127.1.194.84:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:40:24.753750 12456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:24.754040 12456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:24.754081 12456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:24.758148 12456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:24.758203 12456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:24.758299 12456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:40:24.763263 12456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:35761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:24.764384 12456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:24.765861 12456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:24.775748 12461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.775743 12462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.775902 12464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:24.775967 12456 server_base.cc:1034] running on GCE node
I20260808 10:40:24.776508 12456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:24.777175 12456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:24.778340 12456 hybrid_clock.cc:648] HybridClock initialized: now 1786185624778303 us; error 50 us; skew 500 ppm
I20260808 10:40:24.780503 12456 webserver.cc:460] Webserver started at http://127.1.194.65:33751/ using document root <none> and password file <none>
I20260808 10:40:24.781183 12456 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:24.781253 12456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:24.781427 12456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:24.783805 12456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "907189fba7e346bf9344eee787cb3b4c"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "fc45e482cf3e2f58c3b6fde2368f4358"
server_key_iv: "affb8c1389730482ead5bce273f3efc9"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.784425 12456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "907189fba7e346bf9344eee787cb3b4c"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "fc45e482cf3e2f58c3b6fde2368f4358"
server_key_iv: "affb8c1389730482ead5bce273f3efc9"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.788564 12456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:40:24.791117 12470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.792413 12456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:40:24.792517 12456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "907189fba7e346bf9344eee787cb3b4c"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "fc45e482cf3e2f58c3b6fde2368f4358"
server_key_iv: "affb8c1389730482ead5bce273f3efc9"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.792613 12456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:24.797432 12456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:24.798110 12456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:24.798260 12456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:24.798758 12456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:24.799641 12456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:24.799692 12456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.799733 12456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:24.799757 12456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.809412 12456 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:42091
I20260808 10:40:24.809465 12598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:42091 every 8 connection(s)
I20260808 10:40:24.810460 12456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:40:24.813349 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12456
I20260808 10:40:24.813501 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:40:24.813872 1801 external_mini_cluster.cc:1428] Setting key d66fcea8e5140572e99cd7c81ca56972
I20260808 10:40:24.816073 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:35761
--builtin_ntp_servers=127.1.194.84:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 10:40:24.817304 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.812354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:58217 (local address 127.1.194.126:35761)
0808 10:40:24.812508 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:24.812512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:24.813822 (+ 1310us) server_negotiation.cc:407] Connection header received
0808 10:40:24.815374 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:24.815378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:24.815422 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:24.815515 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:24.816734 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:24.816741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:24.816748 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:24.816767 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:24.816795 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:24.816810 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:24.816921 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:24.817048 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:40:24.817101 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:40:24.818275 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.812837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:35761 (local address 127.1.194.65:58217)
0808 10:40:24.813662 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:24.813705 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:24.815119 (+ 1414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:24.815710 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:24.815716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:24.815802 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:24.816421 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:40:24.816423 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:24.816464 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:40:24.816531 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:24.816908 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:24.816914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:24.816918 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:40:24.817048 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 10:40:24.817239 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":165,"threads_started":1}
I20260808 10:40:24.819557 12599 heartbeater.cc:347] Connected to a master server at 127.1.194.126:35761
I20260808 10:40:24.819811 12599 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:24.820319 12599 heartbeater.cc:511] Master 127.1.194.126:35761 requested a full tablet report, sending...
I20260808 10:40:24.822134 12392 ts_manager.cc:194] Registered new tserver with Master: 907189fba7e346bf9344eee787cb3b4c (127.1.194.65:42091)
W20260808 10:40:24.931145 12602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:24.931435 12602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:24.931532 12602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:24.935676 12602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:24.935758 12602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:24.935844 12602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:40:24.940593 12602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:35761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:24.941740 12602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:24.943104 12602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:24.953337 12607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.953320 12610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:24.953508 12608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:24.954339 12602 server_base.cc:1034] running on GCE node
I20260808 10:40:24.954730 12602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:24.955344 12602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:24.956569 12602 hybrid_clock.cc:648] HybridClock initialized: now 1786185624956544 us; error 47 us; skew 500 ppm
I20260808 10:40:24.958554 12602 webserver.cc:460] Webserver started at http://127.1.194.66:38909/ using document root <none> and password file <none>
I20260808 10:40:24.959141 12602 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:24.959245 12602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:24.959474 12602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:24.961537 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "fe092b795378450c94044244afbcf882"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "277366d17a5b507361f2750165ca4b0e"
server_key_iv: "b03073b33c0ddaeabdb8fe02a3cd82cd"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.962126 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "fe092b795378450c94044244afbcf882"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "277366d17a5b507361f2750165ca4b0e"
server_key_iv: "b03073b33c0ddaeabdb8fe02a3cd82cd"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.965749 12602 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:24.968223 12616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.969425 12602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:24.969589 12602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "fe092b795378450c94044244afbcf882"
format_stamp: "Formatted at 2026-08-08 10:40:24 on dist-test-slave-ss4n"
server_key: "277366d17a5b507361f2750165ca4b0e"
server_key_iv: "b03073b33c0ddaeabdb8fe02a3cd82cd"
server_key_version: "encryptionkey@0"
I20260808 10:40:24.969707 12602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:24.974215 12602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:24.974823 12602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:24.975014 12602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:24.975534 12602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:24.976558 12602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:24.976627 12602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.976696 12602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:24.976739 12602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:24.985867 12602 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:35633
I20260808 10:40:24.985891 12740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:35633 every 8 connection(s)
I20260808 10:40:24.986799 12602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:40:24.993057 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.988645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:37441 (local address 127.1.194.126:35761)
0808 10:40:24.988786 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:24.988790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:24.989933 (+ 1143us) server_negotiation.cc:407] Connection header received
0808 10:40:24.991269 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:24.991272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:24.991318 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:24.991429 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:24.992557 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:24.992560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:24.992562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:24.992574 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:24.992595 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:24.992603 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:24.992709 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:24.992856 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 10:40:24.992903 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:40:24.993971 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:24.989036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:35761 (local address 127.1.194.66:37441)
0808 10:40:24.989776 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:24.989835 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:24.991060 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:24.991586 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:24.991592 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:24.991675 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:24.992210 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:40:24.992212 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:24.992251 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:40:24.992317 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:24.992692 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:24.992701 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:24.992707 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:40:24.992887 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 10:40:24.993127 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":117,"threads_started":1}
I20260808 10:40:24.994618 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12602
I20260808 10:40:24.994755 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:40:24.995046 1801 external_mini_cluster.cc:1428] Setting key 0d594cfb50717a594bd85f2b4fe06124
I20260808 10:40:24.995126 12742 heartbeater.cc:347] Connected to a master server at 127.1.194.126:35761
I20260808 10:40:24.995368 12742 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:24.995819 12742 heartbeater.cc:511] Master 127.1.194.126:35761 requested a full tablet report, sending...
I20260808 10:40:24.996915 12392 ts_manager.cc:194] Registered new tserver with Master: fe092b795378450c94044244afbcf882 (127.1.194.66:35633)
I20260808 10:40:24.997268 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:35761
--builtin_ntp_servers=127.1.194.84:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:40:25.110437 12745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:25.110720 12745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:25.110761 12745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:25.114753 12745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:25.114816 12745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:25.114874 12745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:40:25.119678 12745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:35761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:25.120738 12745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:25.122069 12745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:25.132112 12751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:25.132121 12753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:25.132153 12750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:25.132411 12745 server_base.cc:1034] running on GCE node
I20260808 10:40:25.132812 12745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:25.133635 12745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:25.134801 12745 hybrid_clock.cc:648] HybridClock initialized: now 1786185625134775 us; error 37 us; skew 500 ppm
I20260808 10:40:25.136722 12745 webserver.cc:460] Webserver started at http://127.1.194.67:40063/ using document root <none> and password file <none>
I20260808 10:40:25.137368 12745 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:25.137442 12745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:25.137706 12745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:25.139773 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "a07c5bcb0f0542ec94337cf311e77540"
format_stamp: "Formatted at 2026-08-08 10:40:25 on dist-test-slave-ss4n"
server_key: "e1ba16fec81a4c3b311f290085f6e4eb"
server_key_iv: "72ca4772ae3efffe0b677aafe05b2d68"
server_key_version: "encryptionkey@0"
I20260808 10:40:25.140360 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "a07c5bcb0f0542ec94337cf311e77540"
format_stamp: "Formatted at 2026-08-08 10:40:25 on dist-test-slave-ss4n"
server_key: "e1ba16fec81a4c3b311f290085f6e4eb"
server_key_iv: "72ca4772ae3efffe0b677aafe05b2d68"
server_key_version: "encryptionkey@0"
I20260808 10:40:25.144181 12745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:40:25.146819 12760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:25.148072 12745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:40:25.148191 12745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "a07c5bcb0f0542ec94337cf311e77540"
format_stamp: "Formatted at 2026-08-08 10:40:25 on dist-test-slave-ss4n"
server_key: "e1ba16fec81a4c3b311f290085f6e4eb"
server_key_iv: "72ca4772ae3efffe0b677aafe05b2d68"
server_key_version: "encryptionkey@0"
I20260808 10:40:25.148309 12745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:25.153194 12745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:25.153802 12745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:25.153985 12745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:25.154525 12745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:25.155409 12745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:25.155455 12745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:25.155491 12745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:25.155550 12745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:25.165122 12745 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:41835
I20260808 10:40:25.165215 12882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:41835 every 8 connection(s)
I20260808 10:40:25.166121 12745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:40:25.172310 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:25.168003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:52493 (local address 127.1.194.126:35761)
0808 10:40:25.168197 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:25.168200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:25.169226 (+ 1026us) server_negotiation.cc:407] Connection header received
0808 10:40:25.170607 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:25.170611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:25.170658 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:25.170766 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:25.171804 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:25.171808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:25.171810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:40:25.171823 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:25.171845 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:40:25.171861 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:25.171968 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:25.172158 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:40:25.172199 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 10:40:25.173261 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:25.168408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:35761 (local address 127.1.194.67:52493)
0808 10:40:25.169060 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:25.169100 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:25.170365 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:25.170928 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:25.170934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:25.171020 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:25.171549 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:40:25.171550 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:25.171589 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:40:25.171654 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:25.171948 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:25.171956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:25.171962 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:40:25.172177 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 10:40:25.172491 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260808 10:40:25.174468 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12745
I20260808 10:40:25.174590 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:40:25.174772 12885 heartbeater.cc:347] Connected to a master server at 127.1.194.126:35761
I20260808 10:40:25.174996 1801 external_mini_cluster.cc:1428] Setting key cb903cd4e23066111b35032aafdccec1
I20260808 10:40:25.175024 12885 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:25.175467 12885 heartbeater.cc:511] Master 127.1.194.126:35761 requested a full tablet report, sending...
I20260808 10:40:25.176652 12392 ts_manager.cc:194] Registered new tserver with Master: a07c5bcb0f0542ec94337cf311e77540 (127.1.194.67:41835)
I20260808 10:40:25.177384 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:40:25.180375 12434 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:40:25.179573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.1.194.126:35761)
0808 10:40:25.179723 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:25.179727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:25.179859 (+ 132us) server_negotiation.cc:407] Connection header received
0808 10:40:25.179942 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:25.179945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:25.179980 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:25.180045 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:25.180254 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48290: BlockingRecv error: recv got EOF from 127.0.0.1:48290 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:40:25.180488 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12456
I20260808 10:40:25.193701 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12602
I20260808 10:40:25.200258 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12745
I20260808 10:40:25.206235 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12358
2026-08-08T10:40:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (838 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:40:25 dist-test-slave-ss4n krb5kdc[12895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:40:25 dist-test-slave-ss4n krb5kdc[12895](info): set up 2 sockets
Aug 08 10:40:25 dist-test-slave-ss4n krb5kdc[12895](info): commencing operation
krb5kdc: starting...
W20260808 10:40:26.335649 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:40:26 dist-test-slave-ss4n krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:40:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:40:26.465349 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37363
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:26.585078 12910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:26.585330 12910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:26.585372 12910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:26.589336 12910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:40:26.589391 12910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:26.589411 12910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:26.589426 12910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:40:26.589442 12910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:40:26.594740 12910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:37363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:26.595764 12910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:26.596763 12910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:26.605577 12916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:26.605621 12917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:26.605687 12910 server_base.cc:1034] running on GCE node
W20260808 10:40:26.605628 12919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:26.606567 12910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:26.607764 12910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:26.608999 12910 hybrid_clock.cc:648] HybridClock initialized: now 1786185626608984 us; error 35 us; skew 500 ppm
Aug 08 10:40:26 dist-test-slave-ss4n krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:26.611835 12910 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:40:26.612711 12910 webserver.cc:460] Webserver started at http://127.1.194.126:45847/ using document root <none> and password file <none>
I20260808 10:40:26.613277 12910 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:26.613377 12910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:26.613667 12910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:26.615713 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "8fd4a4eced2249749931880b41baf453"
format_stamp: "Formatted at 2026-08-08 10:40:26 on dist-test-slave-ss4n"
server_key: "aa72a19f38429db67c95c02f75d0714e"
server_key_iv: "8c1aab1bedc45dc1cd9b323a63d99c8a"
server_key_version: "encryptionkey@0"
I20260808 10:40:26.616289 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "8fd4a4eced2249749931880b41baf453"
format_stamp: "Formatted at 2026-08-08 10:40:26 on dist-test-slave-ss4n"
server_key: "aa72a19f38429db67c95c02f75d0714e"
server_key_iv: "8c1aab1bedc45dc1cd9b323a63d99c8a"
server_key_version: "encryptionkey@0"
I20260808 10:40:26.620853 12910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:40:26.623432 12927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:26.624629 12910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:26.624776 12910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "8fd4a4eced2249749931880b41baf453"
format_stamp: "Formatted at 2026-08-08 10:40:26 on dist-test-slave-ss4n"
server_key: "aa72a19f38429db67c95c02f75d0714e"
server_key_iv: "8c1aab1bedc45dc1cd9b323a63d99c8a"
server_key_version: "encryptionkey@0"
I20260808 10:40:26.624899 12910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:26.645026 12910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:26.648130 12910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:26.648348 12910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:26.655537 12910 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:37363
I20260808 10:40:26.655840 12981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:37363 every 8 connection(s)
I20260808 10:40:26.656505 12910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:40:26.659468 12982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:26.662909 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 12910
I20260808 10:40:26.663067 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:40:26.663349 1801 external_mini_cluster.cc:1428] Setting key 80588bb51268b79c56bfea055ffa5b64
I20260808 10:40:26.665848 12982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453: Bootstrap starting.
I20260808 10:40:26.668831 12982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:26.669724 12982 log.cc:824] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:26.671985 12982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453: No bootstrap required, opened a new log
Aug 08 10:40:26 dist-test-slave-ss4n krb5kdc[12895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:40:26.676951 12982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd4a4eced2249749931880b41baf453" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37363 } }
I20260808 10:40:26.677268 12982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:26.677320 12982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd4a4eced2249749931880b41baf453, State: Initialized, Role: FOLLOWER
I20260808 10:40:26.677870 12982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd4a4eced2249749931880b41baf453" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37363 } }
I20260808 10:40:26.678050 12982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:40:26.678124 12982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:40:26.678221 12982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:26.679410 12982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd4a4eced2249749931880b41baf453" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37363 } }
I20260808 10:40:26.679680 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:26.664864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.1.194.126:37363)
0808 10:40:26.665297 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:26.665318 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:26.665362 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:40:26.666248 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:26.666271 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:26.666673 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:26.667053 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:26.667750 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:26.669264 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:26.670248 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:26.670843 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:26.671009 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:26.673947 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:26.673953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:26.673971 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:26.674015 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:26.676215 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:26.676721 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:26.676727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:26.676733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:26.676874 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:26.677172 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:26.677175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:26.677177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:26.677779 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:26.678001 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:26.678361 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 10:40:26.678666 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":114,"threads_started":1}
I20260808 10:40:26.679795 12982 leader_election.cc:302] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fd4a4eced2249749931880b41baf453; no voters:
I20260808 10:40:26.680130 12982 leader_election.cc:288] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:40:26.680265 12987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:26.680563 12987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd4a4eced2249749931880b41baf453, State: Running, Role: LEADER
I20260808 10:40:26.680903 12987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd4a4eced2249749931880b41baf453" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37363 } }
I20260808 10:40:26.681565 12982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:40:26.682083 12989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fd4a4eced2249749931880b41baf453. Latest consensus state: current_term: 1 leader_uuid: "8fd4a4eced2249749931880b41baf453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd4a4eced2249749931880b41baf453" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37363 } } }
I20260808 10:40:26.682236 12989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:26.682523 12988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fd4a4eced2249749931880b41baf453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd4a4eced2249749931880b41baf453" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37363 } } }
I20260808 10:40:26.682636 12988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453 [sys.catalog]: This master's current role is: LEADER
I20260808 10:40:26.682785 12992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:40:26.685938 12992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:40:26.691226 12992 catalog_manager.cc:1347] Generated new cluster ID: c7cdf6c962e24a98a859249862958fd6
I20260808 10:40:26.691300 12992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:40:26.712213 12992 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:40:26.713184 12992 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:40:26.720269 12992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453: Generated new TSK 0
I20260808 10:40:26.721051 12992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:40:26.756714 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37363
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:26.883733 13008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:26.883992 13008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:26.884037 13008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:26.887981 13008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:26.888037 13008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:26.888100 13008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:40:26.893558 13008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:26.894690 13008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:26.895722 13008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:26.906129 13015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:26.906136 13014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:26.906128 13017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:26.906406 13008 server_base.cc:1034] running on GCE node
I20260808 10:40:26.907042 13008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:26.907753 13008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:26.908936 13008 hybrid_clock.cc:648] HybridClock initialized: now 1786185626908912 us; error 31 us; skew 500 ppm
Aug 08 10:40:26 dist-test-slave-ss4n krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:26.912168 13008 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:40:26.913182 13008 webserver.cc:460] Webserver started at http://127.1.194.65:45671/ using document root <none> and password file <none>
I20260808 10:40:26.913887 13008 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:26.913959 13008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:26.914297 13008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:26.916618 13008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "70b6e1808c894009a714abb0e37f7615"
format_stamp: "Formatted at 2026-08-08 10:40:26 on dist-test-slave-ss4n"
server_key: "f217e03960f498e85785c048e26ca141"
server_key_iv: "7c64a21d6323dc06a0c6968bc37a49ba"
server_key_version: "encryptionkey@0"
I20260808 10:40:26.917258 13008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "70b6e1808c894009a714abb0e37f7615"
format_stamp: "Formatted at 2026-08-08 10:40:26 on dist-test-slave-ss4n"
server_key: "f217e03960f498e85785c048e26ca141"
server_key_iv: "7c64a21d6323dc06a0c6968bc37a49ba"
server_key_version: "encryptionkey@0"
I20260808 10:40:26.921322 13008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:40:26.923832 13026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:26.925081 13008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:26.925205 13008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "70b6e1808c894009a714abb0e37f7615"
format_stamp: "Formatted at 2026-08-08 10:40:26 on dist-test-slave-ss4n"
server_key: "f217e03960f498e85785c048e26ca141"
server_key_iv: "7c64a21d6323dc06a0c6968bc37a49ba"
server_key_version: "encryptionkey@0"
I20260808 10:40:26.925310 13008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:26.943591 13008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:26.947018 13008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:26.947232 13008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:26.947850 13008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:26.948679 13008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:26.948724 13008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:26.948794 13008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:26.948827 13008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:26.958575 13008 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:44017
I20260808 10:40:26.958629 13147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:44017 every 8 connection(s)
I20260808 10:40:26.959731 13008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:40:26.965181 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 13008
I20260808 10:40:26.965327 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:40:26.965662 1801 external_mini_cluster.cc:1428] Setting key d83dca134adeb2c27dafea62c8468b6b
Aug 08 10:40:26 dist-test-slave-ss4n krb5kdc[12895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:40:26.974059 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:26.961863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:40641 (local address 127.1.194.126:37363)
0808 10:40:26.962092 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:26.962097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:26.962875 (+ 778us) server_negotiation.cc:407] Connection header received
0808 10:40:26.964095 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:26.964098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:26.964141 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:26.964216 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:26.965278 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:26.966059 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:26.967446 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:26.967827 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:26.967952 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:26.970554 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:26.970558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:26.970559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:26.970582 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:26.971989 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:26.972493 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:26.972497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:26.972499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:26.972581 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:26.972925 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:26.972928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:26.972929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:26.973107 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:26.973212 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:26.973760 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 10:40:26.973884 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 10:40:26.974823 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:26.962294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37363 (local address 127.1.194.65:40641)
0808 10:40:26.962726 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:26.962766 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:26.963905 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:26.964336 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:26.964341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:26.964677 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:26.965129 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:26.965142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:26.966192 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:26.966196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:26.967233 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:26.967242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:26.967974 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:26.967976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:26.968111 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:26.968714 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:26.968730 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:26.970382 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:26.972115 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:26.972121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:26.972134 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:26.972389 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:26.972676 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:26.972680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:26.972682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:26.972817 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:26.973210 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:26.973217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:26.973468 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 10:40:26.973719 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 10:40:26.974040 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:40:26.976178 13148 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37363
I20260808 10:40:26.976389 13148 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:26.976867 13148 heartbeater.cc:511] Master 127.1.194.126:37363 requested a full tablet report, sending...
I20260808 10:40:26.978394 12945 ts_manager.cc:194] Registered new tserver with Master: 70b6e1808c894009a714abb0e37f7615 (127.1.194.65:44017)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:40:26.979804 12945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:40641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:40:26.996809 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37363
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:27.106560 13153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:27.106850 13153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:27.106930 13153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:27.110975 13153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:27.111063 13153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:27.111147 13153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:40:27.116438 13153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:27.117511 13153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:27.118507 13153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:27.128129 13159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:27.128198 13161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:27.128309 13158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:27.128512 13153 server_base.cc:1034] running on GCE node
I20260808 10:40:27.128891 13153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:27.129758 13153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:27.130957 13153 hybrid_clock.cc:648] HybridClock initialized: now 1786185627130927 us; error 37 us; skew 500 ppm
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185627, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:27.133831 13153 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:40:27.134644 13153 webserver.cc:460] Webserver started at http://127.1.194.66:41743/ using document root <none> and password file <none>
I20260808 10:40:27.135154 13153 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:27.135212 13153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:27.135370 13153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:27.137390 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "b2bfa6cc44a943da84a082aa629a1f93"
format_stamp: "Formatted at 2026-08-08 10:40:27 on dist-test-slave-ss4n"
server_key: "f8050ddb716f1e6b356f7ee04d02d247"
server_key_iv: "25bc814d09162c9943d34383f991fbeb"
server_key_version: "encryptionkey@0"
I20260808 10:40:27.137975 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "b2bfa6cc44a943da84a082aa629a1f93"
format_stamp: "Formatted at 2026-08-08 10:40:27 on dist-test-slave-ss4n"
server_key: "f8050ddb716f1e6b356f7ee04d02d247"
server_key_iv: "25bc814d09162c9943d34383f991fbeb"
server_key_version: "encryptionkey@0"
I20260808 10:40:27.141443 13153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:27.144243 13168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:27.145565 13153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:40:27.145680 13153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "b2bfa6cc44a943da84a082aa629a1f93"
format_stamp: "Formatted at 2026-08-08 10:40:27 on dist-test-slave-ss4n"
server_key: "f8050ddb716f1e6b356f7ee04d02d247"
server_key_iv: "25bc814d09162c9943d34383f991fbeb"
server_key_version: "encryptionkey@0"
I20260808 10:40:27.145789 13153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:27.172500 13153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:27.175629 13153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:27.175843 13153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:27.176455 13153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:27.177305 13153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:27.177352 13153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:27.177421 13153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:27.177457 13153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:27.187289 13153 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:35015
I20260808 10:40:27.187314 13295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:35015 every 8 connection(s)
I20260808 10:40:27.188318 13153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:40:27.194164 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 13153
I20260808 10:40:27.194288 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:40:27.194635 1801 external_mini_cluster.cc:1428] Setting key d22f27f15b4534411f4554ca6728f86d
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[12895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185627, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:40:27.202834 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.190090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:50237 (local address 127.1.194.126:37363)
0808 10:40:27.190236 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.190241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.191092 (+ 851us) server_negotiation.cc:407] Connection header received
0808 10:40:27.192405 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.192408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.192455 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.192533 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:27.193712 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.194562 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.195875 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.196158 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.196241 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.199137 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:27.199146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:27.199148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:27.199170 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:27.200711 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:27.201220 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:27.201225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:27.201227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:27.201306 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:27.201724 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:27.201728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:27.201730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:27.201914 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:27.202033 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.202529 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.202650 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:40:27.203624 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.190513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37363 (local address 127.1.194.66:50237)
0808 10:40:27.190948 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.190988 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.192206 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.192696 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.192702 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.193048 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:27.193570 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.193585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.194706 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.194713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.195716 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.195749 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.196312 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.196316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.196527 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.197222 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:27.197248 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:27.198973 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:27.200834 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:27.200840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:27.200855 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:27.201115 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:27.201407 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:27.201410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:27.201412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:27.201629 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:27.202022 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:27.202028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:27.202294 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.202472 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.202792 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:40:27.204842 13296 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37363
I20260808 10:40:27.205121 13296 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:27.205636 13296 heartbeater.cc:511] Master 127.1.194.126:37363 requested a full tablet report, sending...
I20260808 10:40:27.206786 12945 ts_manager.cc:194] Registered new tserver with Master: b2bfa6cc44a943da84a082aa629a1f93 (127.1.194.66:35015)
I20260808 10:40:27.207381 12945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:50237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:40:27.225201 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37363
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:40:27.337028 13302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:40:27.337283 13302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:40:27.337327 13302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:40:27.341358 13302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:40:27.341420 13302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:40:27.341562 13302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:40:27.346941 13302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:40:27.348037 13302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:40:27.349110 13302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:40:27.359091 13312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:40:27.359073 13308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:27.359231 13302 server_base.cc:1034] running on GCE node
W20260808 10:40:27.359097 13309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:40:27.359715 13302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:40:27.360401 13302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:40:27.361680 13302 hybrid_clock.cc:648] HybridClock initialized: now 1786185627361654 us; error 32 us; skew 500 ppm
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185627, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:40:27.369264 13302 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:40:27.370241 13302 webserver.cc:460] Webserver started at http://127.1.194.67:44129/ using document root <none> and password file <none>
I20260808 10:40:27.370802 13302 fs_manager.cc:359] Metadata directory not provided
I20260808 10:40:27.370865 13302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:40:27.371050 13302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:40:27.373073 13302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "8231479c666f421283a682acb3bd8a5e"
format_stamp: "Formatted at 2026-08-08 10:40:27 on dist-test-slave-ss4n"
server_key: "6ff00b89b0936ef1e41b37d529848d3c"
server_key_iv: "8b6fcb23135013764c034f871ee0d0c6"
server_key_version: "encryptionkey@0"
I20260808 10:40:27.373646 13302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "8231479c666f421283a682acb3bd8a5e"
format_stamp: "Formatted at 2026-08-08 10:40:27 on dist-test-slave-ss4n"
server_key: "6ff00b89b0936ef1e41b37d529848d3c"
server_key_iv: "8b6fcb23135013764c034f871ee0d0c6"
server_key_version: "encryptionkey@0"
I20260808 10:40:27.377244 13302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:40:27.379768 13321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:27.380980 13302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:40:27.381232 13302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "8231479c666f421283a682acb3bd8a5e"
format_stamp: "Formatted at 2026-08-08 10:40:27 on dist-test-slave-ss4n"
server_key: "6ff00b89b0936ef1e41b37d529848d3c"
server_key_iv: "8b6fcb23135013764c034f871ee0d0c6"
server_key_version: "encryptionkey@0"
I20260808 10:40:27.381361 13302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:40:27.393457 13302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:40:27.396317 13302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:40:27.396518 13302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:40:27.397095 13302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:40:27.398010 13302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:40:27.398057 13302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:27.398167 13302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:40:27.398252 13302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:40:27.408207 13302 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:46413
I20260808 10:40:27.408229 13445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:46413 every 8 connection(s)
I20260808 10:40:27.409137 13302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:40:27.412918 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 13302
I20260808 10:40:27.413097 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:40:27.413431 1801 external_mini_cluster.cc:1428] Setting key 45da21a39ab944dbce311dff03aea716
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[12895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185627, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:40:27.423902 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.410964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:35487 (local address 127.1.194.126:37363)
0808 10:40:27.411121 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.411126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.412012 (+ 886us) server_negotiation.cc:407] Connection header received
0808 10:40:27.413505 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.413509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.413570 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.413672 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:27.415224 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.415892 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.417283 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.417633 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.417727 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.420242 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:27.420245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:27.420247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:27.420270 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:27.421824 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:27.422365 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:27.422370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:27.422372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:27.422454 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:27.422806 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:27.422810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:27.422812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:27.422994 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:27.423098 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.423602 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.423723 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:40:27.424718 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.411425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37363 (local address 127.1.194.67:35487)
0808 10:40:27.411861 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.411902 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.413250 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.413872 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.413880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.414396 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:40:27.415073 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.415102 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.416045 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.416048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.417146 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.417156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.417781 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.417784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.417929 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.418484 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:40:27.418500 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:40:27.420059 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:40:27.421948 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:27.421954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:27.421969 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:27.422257 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:27.422556 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:40:27.422559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:40:27.422561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:40:27.422701 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:40:27.423114 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:40:27.423122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:40:27.423359 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.423552 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.423875 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260808 10:40:27.425988 13446 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37363
I20260808 10:40:27.426261 13446 heartbeater.cc:464] Registering TS with master...
I20260808 10:40:27.426882 13446 heartbeater.cc:511] Master 127.1.194.126:37363 requested a full tablet report, sending...
I20260808 10:40:27.428059 12945 ts_manager.cc:194] Registered new tserver with Master: 8231479c666f421283a682acb3bd8a5e (127.1.194.67:46413)
I20260808 10:40:27.428584 12945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:35487
I20260808 10:40:27.439831 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:40:27.449820 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.442092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.1.194.126:37363)
0808 10:40:27.442264 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.442271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.442438 (+ 167us) server_negotiation.cc:407] Connection header received
0808 10:40:27.442621 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.442623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.442665 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.442753 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:40:27.443520 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.444044 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.444921 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.445147 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.445257 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.446682 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:40:27.446685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:40:27.446688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:40:27.446721 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:40:27.447994 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:27.448420 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:27.448425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:27.448427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:27.448520 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:40:27.448879 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:40:27.448886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:40:27.448888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:40:27.449109 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:40:27.449256 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.449526 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.449651 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:40:27.452876 12945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:40:27.455982 12945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:40:27.470335 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.464456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44017 (local address 127.0.0.1:52244)
0808 10:40:27.464835 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.464886 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.465020 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.465922 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.465930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.465953 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:27.466082 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.466087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.467394 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.467400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.468700 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.468710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.469745 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.469748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.469851 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.469894 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.470028 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.470170 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":24,"thread_start_us":105,"threads_started":1}
I20260808 10:40:27.470340 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.464571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.1.194.65:44017)
0808 10:40:27.465334 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.465352 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.465391 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:40:27.465498 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.465507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.465741 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.466211 (+ 470us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:27.466334 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.467263 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.468825 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.469616 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.469750 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.469938 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.470084 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.470182 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":95,"threads_started":1}
I20260808 10:40:27.470852 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.464801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35015 (local address 127.0.0.1:49164)
0808 10:40:27.465736 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.465752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.465845 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.466407 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.466410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.466426 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:27.466506 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.466511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.467942 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.467945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.469044 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.469055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.470439 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.470442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.470538 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.470557 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.470649 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.470717 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
I20260808 10:40:27.471091 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.465260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.1.194.66:35015)
0808 10:40:27.466071 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.466079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.466097 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:40:27.466154 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.466157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.466279 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.466388 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:27.466633 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.467773 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.469188 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.470149 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.470248 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.470372 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.470682 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.470766 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":135,"threads_started":1}
I20260808 10:40:27.471441 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.465190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:46413 (local address 127.0.0.1:36294)
0808 10:40:27.465776 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.465791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.465879 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.467113 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.467116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.467133 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:27.467208 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.467213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.468562 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.468566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.469571 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.469582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.471079 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.471082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.471165 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.471186 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.471254 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.471317 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":89,"threads_started":1}
I20260808 10:40:27.471902 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.465631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36294 (local address 127.1.194.67:46413)
0808 10:40:27.466688 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.466699 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.466721 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:40:27.466790 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.466794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.466960 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.467120 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:27.467330 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.468417 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.469701 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.470777 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.470882 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.471013 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.471668 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.471771 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":99,"threads_started":1}
I20260808 10:40:27.474290 13371 tablet_service.cc:1467] Processing CreateTablet for tablet 560859a098c44dbd996de814a464afa1 (DEFAULT_TABLE table=test-table [id=9aa3fb3765434d8fbaede6d7e1708b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:27.474225 13079 tablet_service.cc:1467] Processing CreateTablet for tablet 560859a098c44dbd996de814a464afa1 (DEFAULT_TABLE table=test-table [id=9aa3fb3765434d8fbaede6d7e1708b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:27.474228 13223 tablet_service.cc:1467] Processing CreateTablet for tablet 560859a098c44dbd996de814a464afa1 (DEFAULT_TABLE table=test-table [id=9aa3fb3765434d8fbaede6d7e1708b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:40:27.475369 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560859a098c44dbd996de814a464afa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:27.475368 13079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560859a098c44dbd996de814a464afa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:27.475373 13223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560859a098c44dbd996de814a464afa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:40:27.481449 13464 tablet_bootstrap.cc:492] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Bootstrap starting.
I20260808 10:40:27.482224 13465 tablet_bootstrap.cc:492] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Bootstrap starting.
I20260808 10:40:27.483211 13466 tablet_bootstrap.cc:492] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615: Bootstrap starting.
I20260808 10:40:27.485004 13466 tablet_bootstrap.cc:654] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:27.485100 13465 tablet_bootstrap.cc:654] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:27.485179 13464 tablet_bootstrap.cc:654] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Neither blocks nor log segments found. Creating new log.
I20260808 10:40:27.485929 13466 log.cc:824] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:27.486034 13464 log.cc:824] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:27.486034 13465 log.cc:824] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:40:27.487982 13465 tablet_bootstrap.cc:492] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: No bootstrap required, opened a new log
I20260808 10:40:27.487990 13466 tablet_bootstrap.cc:492] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615: No bootstrap required, opened a new log
I20260808 10:40:27.487987 13464 tablet_bootstrap.cc:492] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: No bootstrap required, opened a new log
I20260808 10:40:27.488197 13464 ts_tablet_manager.cc:1397] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260808 10:40:27.488197 13466 ts_tablet_manager.cc:1397] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:40:27.488198 13465 ts_tablet_manager.cc:1397] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260808 10:40:27.491204 13465 raft_consensus.cc:348] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.491451 13465 raft_consensus.cc:374] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:27.491504 13465 raft_consensus.cc:729] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8231479c666f421283a682acb3bd8a5e, State: Initialized, Role: FOLLOWER
I20260808 10:40:27.491920 13465 consensus_queue.cc:260] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.492058 13464 raft_consensus.cc:348] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.492306 13464 raft_consensus.cc:374] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:27.492365 13464 raft_consensus.cc:729] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2bfa6cc44a943da84a082aa629a1f93, State: Initialized, Role: FOLLOWER
I20260808 10:40:27.492487 13466 raft_consensus.cc:348] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.492702 13446 heartbeater.cc:503] Master 127.1.194.126:37363 was elected leader, sending a full tablet report...
I20260808 10:40:27.492715 13466 raft_consensus.cc:374] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:40:27.492760 13466 raft_consensus.cc:729] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b6e1808c894009a714abb0e37f7615, State: Initialized, Role: FOLLOWER
I20260808 10:40:27.492763 13464 consensus_queue.cc:260] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.492987 13465 ts_tablet_manager.cc:1428] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260808 10:40:27.493186 13466 consensus_queue.cc:260] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.493840 13148 heartbeater.cc:503] Master 127.1.194.126:37363 was elected leader, sending a full tablet report...
I20260808 10:40:27.494068 13466 ts_tablet_manager.cc:1428] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:40:27.494138 13296 heartbeater.cc:503] Master 127.1.194.126:37363 was elected leader, sending a full tablet report...
I20260808 10:40:27.495339 13464 ts_tablet_manager.cc:1428] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260808 10:40:27.612277 13472 raft_consensus.cc:482] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:40:27.612510 13472 raft_consensus.cc:504] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.613843 13472 leader_election.cc:288] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70b6e1808c894009a714abb0e37f7615 (127.1.194.65:44017), 8231479c666f421283a682acb3bd8a5e (127.1.194.67:46413)
I20260808 10:40:27.618383 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.614129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44017 (local address 127.1.194.66:40457)
0808 10:40:27.614258 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.614274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.614382 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.614776 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.614782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.614804 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:27.614918 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.614946 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.615932 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.615935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.617098 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.617108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.618037 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.618040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.618132 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.618152 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.618221 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.618275 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:40:27.618495 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.614248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:40457 (local address 127.1.194.65:44017)
0808 10:40:27.614420 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.614426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.614447 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:40:27.614574 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.614577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.614648 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.614747 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:27.615083 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.615801 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.617239 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.617919 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.618056 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.618110 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.618259 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.618357 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:40:27.618917 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.614440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:46413 (local address 127.1.194.66:49443)
0808 10:40:27.614856 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.614879 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:40:27.614966 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:40:27.615257 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:40:27.615259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:40:27.615281 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:40:27.615394 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.615400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.616388 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.616392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.617561 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:40:27.617571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.618514 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.618518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:40:27.618614 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.618648 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 10:40:27.618718 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:40:27.618776 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":11,"thread_start_us":155,"threads_started":1}
I20260808 10:40:27.618924 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.614615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:49443 (local address 127.1.194.67:46413)
0808 10:40:27.614761 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.614765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.614962 (+ 197us) server_negotiation.cc:407] Connection header received
0808 10:40:27.615097 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.615099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.615139 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.615196 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:40:27.615577 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.616253 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.617725 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.618340 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.618463 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.618519 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.618738 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.618814 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:40:27.619115 13099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560859a098c44dbd996de814a464afa1" candidate_uuid: "b2bfa6cc44a943da84a082aa629a1f93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b6e1808c894009a714abb0e37f7615" is_pre_election: true
I20260808 10:40:27.619454 13099 raft_consensus.cc:2454] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2bfa6cc44a943da84a082aa629a1f93 in term 0.
I20260808 10:40:27.619724 13394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560859a098c44dbd996de814a464afa1" candidate_uuid: "b2bfa6cc44a943da84a082aa629a1f93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8231479c666f421283a682acb3bd8a5e" is_pre_election: true
I20260808 10:40:27.619952 13170 leader_election.cc:302] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70b6e1808c894009a714abb0e37f7615, b2bfa6cc44a943da84a082aa629a1f93; no voters:
I20260808 10:40:27.620015 13394 raft_consensus.cc:2454] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2bfa6cc44a943da84a082aa629a1f93 in term 0.
I20260808 10:40:27.620285 13472 raft_consensus.cc:2781] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:40:27.620388 13472 raft_consensus.cc:482] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:40:27.620445 13472 raft_consensus.cc:3037] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:27.621555 13472 raft_consensus.cc:504] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.621898 13472 leader_election.cc:288] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [CANDIDATE]: Term 1 election: Requested vote from peers 70b6e1808c894009a714abb0e37f7615 (127.1.194.65:44017), 8231479c666f421283a682acb3bd8a5e (127.1.194.67:46413)
I20260808 10:40:27.622278 13099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560859a098c44dbd996de814a464afa1" candidate_uuid: "b2bfa6cc44a943da84a082aa629a1f93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b6e1808c894009a714abb0e37f7615"
I20260808 10:40:27.622411 13099 raft_consensus.cc:3037] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:27.622424 13394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560859a098c44dbd996de814a464afa1" candidate_uuid: "b2bfa6cc44a943da84a082aa629a1f93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8231479c666f421283a682acb3bd8a5e"
I20260808 10:40:27.622615 13394 raft_consensus.cc:3037] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:40:27.623483 13099 raft_consensus.cc:2454] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2bfa6cc44a943da84a082aa629a1f93 in term 1.
I20260808 10:40:27.623713 13394 raft_consensus.cc:2454] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2bfa6cc44a943da84a082aa629a1f93 in term 1.
I20260808 10:40:27.623780 13170 leader_election.cc:302] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70b6e1808c894009a714abb0e37f7615, b2bfa6cc44a943da84a082aa629a1f93; no voters:
I20260808 10:40:27.623944 13472 raft_consensus.cc:2781] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:40:27.624185 13472 raft_consensus.cc:686] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 1 LEADER]: Becoming Leader. State: Replica: b2bfa6cc44a943da84a082aa629a1f93, State: Running, Role: LEADER
I20260808 10:40:27.624516 13472 consensus_queue.cc:237] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } }
I20260808 10:40:27.628132 12944 catalog_manager.cc:5495] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2bfa6cc44a943da84a082aa629a1f93 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "b2bfa6cc44a943da84a082aa629a1f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2bfa6cc44a943da84a082aa629a1f93" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35015 } health_report { overall_health: HEALTHY } } }
W20260808 10:40:27.660224 13447 tablet.cc:2369] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:40:27.663133 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.659374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.1.194.66:35015)
0808 10:40:27.659516 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:40:27.659523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:40:27.659600 (+ 77us) server_negotiation.cc:407] Connection header received
0808 10:40:27.659674 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:40:27.659676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:40:27.659731 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:40:27.659843 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:40:27.660110 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.660727 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.661678 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:40:27.661899 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:40:27.662011 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:40:27.662347 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:40:27.662735 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:40:27.662870 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:40:27.662932 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 10:40:27.662989 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:40:27.669217 13099 raft_consensus.cc:1261] T 560859a098c44dbd996de814a464afa1 P 70b6e1808c894009a714abb0e37f7615 [term 1 FOLLOWER]: Refusing update from remote peer b2bfa6cc44a943da84a082aa629a1f93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:27.669217 13394 raft_consensus.cc:1261] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 1 FOLLOWER]: Refusing update from remote peer b2bfa6cc44a943da84a082aa629a1f93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:40:27.670135 13472 consensus_queue.cc:1035] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70b6e1808c894009a714abb0e37f7615" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 44017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:27.670370 13475 consensus_queue.cc:1035] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8231479c666f421283a682acb3bd8a5e" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:40:27.679122 13480 mvcc.cc:205] Tried to move back new op lower bound from 7316216330925584384 to 7316216330752368640. Current Snapshot: MvccSnapshot[applied={T|T < 7316216330925584384}]
I20260808 10:40:27.679936 13479 mvcc.cc:205] Tried to move back new op lower bound from 7316216330925584384 to 7316216330752368640. Current Snapshot: MvccSnapshot[applied={T|T < 7316216330925584384}]
I20260808 10:40:27.688274 12945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:27.688476 12945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:40:27.690991 12945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fd4a4eced2249749931880b41baf453: Sending DeleteTablet for 3 replicas of tablet 560859a098c44dbd996de814a464afa1
I20260808 10:40:27.691848 13223 tablet_service.cc:1514] Processing DeleteTablet for tablet 560859a098c44dbd996de814a464afa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:27 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:49164
I20260808 10:40:27.691844 13079 tablet_service.cc:1514] Processing DeleteTablet for tablet 560859a098c44dbd996de814a464afa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:27 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:52244
I20260808 10:40:27.691975 13371 tablet_service.cc:1514] Processing DeleteTablet for tablet 560859a098c44dbd996de814a464afa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:40:27 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:36294
I20260808 10:40:27.692791 13492 tablet_replica.cc:330] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: stopping tablet replica
I20260808 10:40:27.692823 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 13008
I20260808 10:40:27.694648 13491 tablet_replica.cc:330] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: stopping tablet replica
I20260808 10:40:27.694952 13491 raft_consensus.cc:2229] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:40:27.695355 13491 raft_consensus.cc:2258] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:27.695685 13492 raft_consensus.cc:2229] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:40:27.696000 13492 raft_consensus.cc:2258] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:40:27.697408 13491 ts_tablet_manager.cc:1905] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:27.697623 13492 ts_tablet_manager.cc:1905] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:40:27.699640 13491 ts_tablet_manager.cc:1918] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:27.699761 13491 log.cc:1196] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/560859a098c44dbd996de814a464afa1
I20260808 10:40:27.700042 13491 ts_tablet_manager.cc:1939] T 560859a098c44dbd996de814a464afa1 P b2bfa6cc44a943da84a082aa629a1f93: Deleting consensus metadata
I20260808 10:40:27.700292 13492 ts_tablet_manager.cc:1918] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:40:27.700387 13492 log.cc:1196] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/560859a098c44dbd996de814a464afa1
I20260808 10:40:27.700713 13492 ts_tablet_manager.cc:1939] T 560859a098c44dbd996de814a464afa1 P 8231479c666f421283a682acb3bd8a5e: Deleting consensus metadata
I20260808 10:40:27.701201 12931 catalog_manager.cc:4841] TS b2bfa6cc44a943da84a082aa629a1f93 (127.1.194.66:35015): tablet 560859a098c44dbd996de814a464afa1 (table test-table [id=9aa3fb3765434d8fbaede6d7e1708b26]) successfully deleted
I20260808 10:40:27.701855 12928 catalog_manager.cc:4841] TS 8231479c666f421283a682acb3bd8a5e (127.1.194.67:46413): tablet 560859a098c44dbd996de814a464afa1 (table test-table [id=9aa3fb3765434d8fbaede6d7e1708b26]) successfully deleted
W20260808 10:40:27.706249 12928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:44017 (error 108)
I20260808 10:40:27.706368 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 13153
I20260808 10:40:27.708784 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:40:27.708382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:44017 (local address 127.0.0.1:52248)
0808 10:40:27.708595 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 10:40:27.708672 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:44017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 10:40:27.708969 12928 catalog_manager.cc:4584] TS 70b6e1808c894009a714abb0e37f7615 (127.1.194.65:44017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 560859a098c44dbd996de814a464afa1: Network error: Client connection negotiation failed: client connection to 127.1.194.65:44017: connect: Connection refused (error 111)
I20260808 10:40:27.713639 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 13302
I20260808 10:40:27.720973 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 12910
2026-08-08T10:40:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2515 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[13496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[13496](info): set up 2 sockets
Aug 08 10:40:27 dist-test-slave-ss4n krb5kdc[13496](info): commencing operation
krb5kdc: starting...
W20260808 10:40:28.860884 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:40:28 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:40:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:40:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/rangeradmin_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/rangeradmin_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/rangerlookup_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/rangerlookup_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
I20260808 10:40:29.054100 1801 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBjlYe1/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/postgres -l logfile start
2026-08-08 10:40:31.858 UTC [13529] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:40:31.858 UTC [13529] LOG: listening on IPv4 address "127.1.194.84", port 38903
2026-08-08 10:40:31.862 UTC [13529] LOG: listening on Unix socket "/tmp/.s.PGSQL.38903"
2026-08-08 10:40:31.885 UTC [13532] LOG: database system was shut down at 2026-08-08 10:40:30 UTC
2026-08-08 10:40:31.889 UTC [13529] LOG: database system is ready to accept connections
I20260808 10:40:32.896339 1801 mini_postgres.cc:96] Postgres bound to 38903
2026-08-08 10:40:32.904 UTC [13541] FATAL: database "slave" does not exist
127.1.194.84:38903 - accepting connections
I20260808 10:40:32.905139 1801 mini_ranger.cc:161] Starting Ranger...
I20260808 10:40:32.921929 1801 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:40:33.482432 1801 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:40:33.482548 1801 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-admin
2026-08-08 10:40:33,817 [I] DB FLAVOR :POSTGRES
2026-08-08 10:40:33,818 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:40:33,818 [I] Checking connection..
2026-08-08 10:40:33,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:40:34,083 [I] Checking connection passed.
2026-08-08 10:40:34,083 [I] --------- Verifying version history table ---------
2026-08-08 10:40:34,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:40:34,355 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:40:34,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:40:34,621 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:40:34,621 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:40:34,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:40:34.904 UTC [13609] WARNING: there is no transaction in progress
2026-08-08 10:40:34,911 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:40:34,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:40:35,172 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:40:35,173 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:40:35,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:40:35,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:40:35,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-08 10:40:35,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:40:36,239 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:40:36,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:40:36,617 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:40:36,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:40:36,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:40:37,157 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:40:37,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:40:38.190 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.207 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.225 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.243 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.260 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.279 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.339 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.346 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.362 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.375 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.385 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.400 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.410 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.418 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.425 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:38.433 UTC [13727] WARNING: there is no transaction in progress
2026-08-08 10:40:39,167 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:40:39,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:40:39,409 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:40:39,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:40:39,678 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:40:39,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:40:39,954 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:40:39,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:40:40,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:40:40,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-08 10:40:40,740 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:40:40,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:40:41,008 [I] DB_PATCHES have already been applied
I20260808 10:40:41.015480 1801 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBjlYe1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:40:41.015549 1801 mini_ranger.cc:193] Using host: 127.1.194.84
I20260808 10:40:41.019968 1801 mini_ranger.cc:227] Ranger admin URL: http://127.1.194.84:39423
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:40:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:40:42.195637 1801 mini_ranger.cc:160] Time spent starting Ranger: real 9.290s user 0.000s sys 0.013s
Aug 08, 2026 10:41:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:41:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:41:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:41:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:41:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:41:05.044452 1801 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/rangerkms_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/rangerkms_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.194.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.194.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:41:05 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185665, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:41:05.139914 1801 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:41:05.150923 1801 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:41:06.151664 1801 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:41:06.151769 1801 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms
I20260808 10:41:06.151877 1801 mini_ranger_kms.cc:227] Using postgres at 127.1.194.84:38903
2026-08-08 10:41:06,476 [I] DB FLAVOR :POSTGRES
2026-08-08 10:41:06,476 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:41:06,476 [I] Checking connection
2026-08-08 10:41:06,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:41:06,762 [I] connection success
2026-08-08 10:41:06,763 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:41:06,763 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:41:06,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:41:07,055 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:41:07,056 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:41:07,056 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:41:07,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:38903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:41:07,323 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:41:07.825147 1801 mini_ranger_kms.cc:310] Created kms service
I20260808 10:41:08.054877 1801 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:41:08.105170 1801 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:41:08 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185665, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:08.525054 1801 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:41:08.525249 1801 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:41:08.525291 1801 mini_ranger_kms.cc:240] Using host: 127.1.194.84
I20260808 10:41:08.530745 1801 mini_ranger_kms.cc:276] Ranger KMS PID: 13924
I20260808 10:41:08.530898 1801 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.194.84:60187
Aug 08, 2026 10:41:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:41:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:41:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:41:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:41:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:41:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 10:41:11.029920 1801 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.890s user 0.024s sys 0.280s
I20260808 10:41:11.030078 1801 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:41:11.030196 1801 mini_ranger_kms.cc:401] 127.1.194.84:60187/kms/v1/keys
Aug 08 10:41:11 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185671, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.194.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 10:41:11 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185671, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.194.84@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
Aug 08 10:41:11 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185671, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.194.84@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
Aug 08 10:41:14 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:41:14.382238 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45393
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45393
--ranger_config_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:60187/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:41:14.509354 13977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:41:14.509696 13977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:41:14.509769 13977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:41:14.513928 13977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:41:14.513995 13977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:41:14.514016 13977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:41:14.514065 13977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:41:14.514096 13977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:41:14.519814 13977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.194.84:60187/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45393
--ranger_config_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:41:14.520969 13977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:41:14.522303 13977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:41:14.531962 13985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:14.531917 13982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:14.531929 13983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:41:14.532260 13977 server_base.cc:1034] running on GCE node
I20260808 10:41:14.532888 13977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:41:14.534117 13977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:41:14.535298 13977 hybrid_clock.cc:648] HybridClock initialized: now 1786185674535276 us; error 42 us; skew 500 ppm
Aug 08 10:41:14 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:41:14.538281 13977 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:41:14.539280 13977 webserver.cc:460] Webserver started at http://127.1.194.126:36985/ using document root <none> and password file <none>
I20260808 10:41:14.539887 13977 fs_manager.cc:359] Metadata directory not provided
I20260808 10:41:14.539943 13977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:41:14.540161 13977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:41:14 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:14.596963 13977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "448e71402c4449c892e8a7401215b3ca"
format_stamp: "Formatted at 2026-08-08 10:41:14 on dist-test-slave-ss4n"
server_key: "0134b27deff79acfa0650c66681462d3"
server_key_iv: "d2f66b2324bbc5878fd2cbc130b5e2ab"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:14.597756 13977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "448e71402c4449c892e8a7401215b3ca"
format_stamp: "Formatted at 2026-08-08 10:41:14 on dist-test-slave-ss4n"
server_key: "0134b27deff79acfa0650c66681462d3"
server_key_iv: "d2f66b2324bbc5878fd2cbc130b5e2ab"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:14.602265 13977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:41:14.619349 13992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:14.620895 13977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:41:14.621022 13977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "448e71402c4449c892e8a7401215b3ca"
format_stamp: "Formatted at 2026-08-08 10:41:14 on dist-test-slave-ss4n"
server_key: "0134b27deff79acfa0650c66681462d3"
server_key_iv: "d2f66b2324bbc5878fd2cbc130b5e2ab"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:14.621215 13977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:41:14.639020 13977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:41:14.642297 13977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:41:14.642608 13977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:41:14.655220 13977 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:45393
I20260808 10:41:14.655238 14049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:45393 every 8 connection(s)
I20260808 10:41:14.656378 13977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:41:14.659597 14050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:41:14.662681 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 13977
I20260808 10:41:14.662824 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:41:14.663125 1801 external_mini_cluster.cc:1428] Setting key 2b1e9857c5ddb0e58a4f264c423e48f9
I20260808 10:41:14.666862 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca: Bootstrap starting.
I20260808 10:41:14.669976 14050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca: Neither blocks nor log segments found. Creating new log.
I20260808 10:41:14.670856 14050 log.cc:824] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca: Log is configured to *not* fsync() on all Append() calls
I20260808 10:41:14.673334 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca: No bootstrap required, opened a new log
Aug 08 10:41:14 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:41:14.678706 14050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448e71402c4449c892e8a7401215b3ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45393 } }
I20260808 10:41:14.679132 14050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:41:14.679222 14050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448e71402c4449c892e8a7401215b3ca, State: Initialized, Role: FOLLOWER
I20260808 10:41:14.679786 14050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448e71402c4449c892e8a7401215b3ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45393 } }
I20260808 10:41:14.679984 14050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:41:14.680119 14050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:41:14.680239 14050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:41:14.681703 14050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448e71402c4449c892e8a7401215b3ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45393 } }
I20260808 10:41:14.682108 14050 leader_election.cc:302] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 448e71402c4449c892e8a7401215b3ca; no voters:
I20260808 10:41:14.682508 14050 leader_election.cc:288] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:41:14.682632 14055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:41:14.682662 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:14.664667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.1.194.126:45393)
0808 10:41:14.665317 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:14.665370 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:14.665422 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:41:14.666459 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:14.666483 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:14.666931 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:14.667316 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:41:14.669061 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:14.670909 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:14.672242 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:14.672979 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:14.673154 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:14.676590 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:41:14.676599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:41:14.676617 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:41:14.676653 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:41:14.678917 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:14.679416 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:14.679422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:14.679428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:14.679580 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:14.679894 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:14.679897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:14.679899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:14.680497 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:41:14.680726 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:14.681014 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 10:41:14.681580 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":205,"threads_started":1}
I20260808 10:41:14.683126 14055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [term 1 LEADER]: Becoming Leader. State: Replica: 448e71402c4449c892e8a7401215b3ca, State: Running, Role: LEADER
I20260808 10:41:14.683523 14055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448e71402c4449c892e8a7401215b3ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45393 } }
I20260808 10:41:14.684150 14050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:41:14.685047 14055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "448e71402c4449c892e8a7401215b3ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448e71402c4449c892e8a7401215b3ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45393 } } }
I20260808 10:41:14.685267 14055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [sys.catalog]: This master's current role is: LEADER
I20260808 10:41:14.685557 14055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 448e71402c4449c892e8a7401215b3ca. Latest consensus state: current_term: 1 leader_uuid: "448e71402c4449c892e8a7401215b3ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448e71402c4449c892e8a7401215b3ca" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45393 } } }
I20260808 10:41:14.685701 14055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca [sys.catalog]: This master's current role is: LEADER
I20260808 10:41:14.686594 14065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:41:14.686965 14050 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:41:14.694303 14065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:41:14.703819 14065 catalog_manager.cc:1347] Generated new cluster ID: e7c6cf44af084f0cb196bfd2e67a0a6e
I20260808 10:41:14.703896 14065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:41:14.730583 14065 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:41:14.732000 14065 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:41:14.745123 14065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca: Generated new TSK 0
I20260808 10:41:14.746156 14065 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:41:15.645258 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:60187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45393
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:41:15.796664 14098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:41:15.796921 14098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:41:15.796965 14098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:41:15.801386 14098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:41:15.801448 14098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:41:15.801577 14098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:41:15.807349 14098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.194.84:60187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:41:15.808511 14098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:41:15.809572 14098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:41:15.820798 14107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:15.820886 14103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:15.821139 14105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:41:15.821237 14098 server_base.cc:1034] running on GCE node
I20260808 10:41:15.821995 14098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:41:15.822768 14098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:41:15.823943 14098 hybrid_clock.cc:648] HybridClock initialized: now 1786185675823926 us; error 36 us; skew 500 ppm
Aug 08 10:41:15 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185675, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:41:15.827095 14098 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:41:15.828184 14098 webserver.cc:460] Webserver started at http://127.1.194.65:46213/ using document root <none> and password file <none>
I20260808 10:41:15.828855 14098 fs_manager.cc:359] Metadata directory not provided
I20260808 10:41:15.828923 14098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:41:15.829144 14098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:41:15 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185675, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:15.849808 14098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "5141a87c81f44392802a66b59c42d816"
format_stamp: "Formatted at 2026-08-08 10:41:15 on dist-test-slave-ss4n"
server_key: "e46f9ba96bfd24a8b392d2dedee48175"
server_key_iv: "f43c28dc0c5642b14a5c1518559b4185"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:15.850492 14098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "5141a87c81f44392802a66b59c42d816"
format_stamp: "Formatted at 2026-08-08 10:41:15 on dist-test-slave-ss4n"
server_key: "e46f9ba96bfd24a8b392d2dedee48175"
server_key_iv: "f43c28dc0c5642b14a5c1518559b4185"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:15.855329 14098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:41:15.873864 14116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:15.875624 14098 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 10:41:15.875801 14098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "5141a87c81f44392802a66b59c42d816"
format_stamp: "Formatted at 2026-08-08 10:41:15 on dist-test-slave-ss4n"
server_key: "e46f9ba96bfd24a8b392d2dedee48175"
server_key_iv: "f43c28dc0c5642b14a5c1518559b4185"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:15.875926 14098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:41:15.909008 14098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:41:15.912321 14098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:41:15.912566 14098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:41:15.913450 14098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:41:15.914804 14098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:41:15.914897 14098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:15.914985 14098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:41:15.915035 14098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:15.930820 14098 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:38249
I20260808 10:41:15.932636 14098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:41:15.933795 14236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:38249 every 8 connection(s)
I20260808 10:41:15.939579 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 14098
I20260808 10:41:15.939728 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:41:15.940079 1801 external_mini_cluster.cc:1428] Setting key ce45b18341d70e8299b8f8f4f4ceab5f
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Aug 08 10:41:15 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185675, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:41:15.968961 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:15.935063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:34565 (local address 127.1.194.126:45393)
0808 10:41:15.937713 (+ 2650us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:15.937718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:15.950973 (+ 13255us) server_negotiation.cc:407] Connection header received
0808 10:41:15.953622 (+ 2649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:15.953626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:15.953692 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:15.953764 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:41:15.955854 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:15.956743 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:15.958546 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:15.958922 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:15.959007 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:15.964503 (+ 5496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:41:15.964507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:41:15.964510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:41:15.964545 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:41:15.966574 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:15.967128 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:15.967136 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:15.967137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:15.967216 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:15.967579 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:15.967585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:15.967590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:15.967850 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:41:15.967958 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:15.968526 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 10:41:15.968676 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2551,"thread_start_us":92,"threads_started":1}
I20260808 10:41:15.969924 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:15.935598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45393 (local address 127.1.194.65:34565)
0808 10:41:15.950745 (+ 15147us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:15.950837 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:15.952261 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:15.954228 (+ 1967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:15.954238 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:15.954826 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:41:15.955663 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:15.955686 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:15.956908 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:15.956911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:15.958394 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:15.958405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:15.959234 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:15.959237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:15.959486 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:15.960238 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:41:15.960259 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:41:15.964229 (+ 3970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:41:15.966706 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:41:15.966713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:41:15.966730 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:41:15.967001 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:41:15.967319 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:41:15.967322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:41:15.967324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:41:15.967472 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:41:15.967986 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:41:15.967993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:41:15.968250 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 10:41:15.968425 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 10:41:15.968726 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14815,"thread_start_us":152,"threads_started":1}
I20260808 10:41:15.971776 14237 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45393
I20260808 10:41:15.972151 14237 heartbeater.cc:464] Registering TS with master...
I20260808 10:41:15.972895 14237 heartbeater.cc:511] Master 127.1.194.126:45393 requested a full tablet report, sending...
I20260808 10:41:15.974723 14009 ts_manager.cc:194] Registered new tserver with Master: 5141a87c81f44392802a66b59c42d816 (127.1.194.65:38249)
I20260808 10:41:15.976112 14009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:34565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:41:15.987001 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:60187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45393
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 10:41:16.185370 14245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:41:16.185797 14245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:41:16.185936 14245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:41:16.192988 14245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:41:16.193141 14245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:41:16.193287 14245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:41:16.202211 14245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.194.84:60187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:41:16.203936 14245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:41:16.205427 14245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:41:16.217598 14251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:16.217602 14252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:16.217710 14255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:41:16.218226 14245 server_base.cc:1034] running on GCE node
I20260808 10:41:16.218808 14245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:41:16.219743 14245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:41:16.220990 14245 hybrid_clock.cc:648] HybridClock initialized: now 1786185676220965 us; error 41 us; skew 500 ppm
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:41:16.224931 14245 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:41:16.226155 14245 webserver.cc:460] Webserver started at http://127.1.194.66:45609/ using document root <none> and password file <none>
I20260808 10:41:16.226852 14245 fs_manager.cc:359] Metadata directory not provided
I20260808 10:41:16.226948 14245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:41:16.227156 14245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:16.243667 14245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "961967a608234c0e93c7ae363c762a75"
format_stamp: "Formatted at 2026-08-08 10:41:16 on dist-test-slave-ss4n"
server_key: "686fbaa79c024cc554ca9773e31750fd"
server_key_iv: "2bbd5011bcd27773b05010b5feb71655"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:16.244573 14245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "961967a608234c0e93c7ae363c762a75"
format_stamp: "Formatted at 2026-08-08 10:41:16 on dist-test-slave-ss4n"
server_key: "686fbaa79c024cc554ca9773e31750fd"
server_key_iv: "2bbd5011bcd27773b05010b5feb71655"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:16.251119 14245 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260808 10:41:16.269658 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:16.273059 14245 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260808 10:41:16.273310 14245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "961967a608234c0e93c7ae363c762a75"
format_stamp: "Formatted at 2026-08-08 10:41:16 on dist-test-slave-ss4n"
server_key: "686fbaa79c024cc554ca9773e31750fd"
server_key_iv: "2bbd5011bcd27773b05010b5feb71655"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:16.273547 14245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:41:16.293809 14245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:41:16.302901 14245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:41:16.303205 14245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:41:16.304013 14245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:41:16.305094 14245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:41:16.305157 14245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:16.305210 14245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:41:16.305238 14245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:16.320817 14245 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:44893
I20260808 10:41:16.320847 14383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:44893 every 8 connection(s)
I20260808 10:41:16.321904 14245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:41:16.322101 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 14245
I20260808 10:41:16.322234 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:41:16.322602 1801 external_mini_cluster.cc:1428] Setting key 4245908db62866ef7ee0bd59c93d7ad7
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:41:16.345803 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.326499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:60441 (local address 127.1.194.126:45393)
0808 10:41:16.326644 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.326651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.327486 (+ 835us) server_negotiation.cc:407] Connection header received
0808 10:41:16.329137 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.329144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.329209 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.329267 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:41:16.331220 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.332158 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.334123 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.334539 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.334634 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.340642 (+ 6008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:41:16.340650 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:41:16.340656 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:41:16.340693 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:41:16.342857 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:16.343606 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:16.343612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:16.343617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:16.343708 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:16.344163 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:16.344168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:16.344173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:16.344451 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:41:16.344551 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.345366 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.345551 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:41:16.347769 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.325265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45393 (local address 127.1.194.66:60441)
0808 10:41:16.327309 (+ 2044us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.327371 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.328884 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.329731 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.329743 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.330269 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:41:16.331035 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.331057 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.332348 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.332352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.333969 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.333984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.334876 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.334880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.335107 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.336024 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:41:16.336048 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:41:16.340352 (+ 4304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:41:16.343025 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:41:16.343033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:41:16.343054 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:41:16.343465 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:41:16.343836 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:41:16.343842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:41:16.343847 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:41:16.344047 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:41:16.344634 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:41:16.344644 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:41:16.345027 (+ 383us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.346088 (+ 1061us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.346494 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1782,"thread_start_us":260,"threads_started":1}
I20260808 10:41:16.349197 14384 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45393
I20260808 10:41:16.349462 14384 heartbeater.cc:464] Registering TS with master...
I20260808 10:41:16.350153 14384 heartbeater.cc:511] Master 127.1.194.126:45393 requested a full tablet report, sending...
I20260808 10:41:16.351351 14009 ts_manager.cc:194] Registered new tserver with Master: 961967a608234c0e93c7ae363c762a75 (127.1.194.66:44893)
I20260808 10:41:16.352097 14009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:60441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:41:16.361863 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:60187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45393
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:41:16.522590 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:41:16.523000 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:41:16.523088 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:41:16.529227 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:41:16.529316 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:41:16.529425 14389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:41:16.537603 14389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.194.84:60187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:41:16.538965 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:41:16.540630 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:41:16.555256 14397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:16.555660 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:41:16.555795 14395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:41:16.557204 14389 server_base.cc:1034] running on GCE node
I20260808 10:41:16.557723 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:41:16.558502 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:41:16.559706 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786185676559685 us; error 43 us; skew 500 ppm
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:41:16.563977 14389 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:41:16.565235 14389 webserver.cc:460] Webserver started at http://127.1.194.67:32841/ using document root <none> and password file <none>
I20260808 10:41:16.565996 14389 fs_manager.cc:359] Metadata directory not provided
I20260808 10:41:16.566092 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:41:16.566332 14389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:16.591886 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "c5cb45edc3e2403c9239fbd9856d75f7"
format_stamp: "Formatted at 2026-08-08 10:41:16 on dist-test-slave-ss4n"
server_key: "c065796f4b3cc96cccf9fc45efd0a841"
server_key_iv: "f2433047649f39f028e66b560a3acb8e"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:16.592612 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "c5cb45edc3e2403c9239fbd9856d75f7"
format_stamp: "Formatted at 2026-08-08 10:41:16 on dist-test-slave-ss4n"
server_key: "c065796f4b3cc96cccf9fc45efd0a841"
server_key_iv: "f2433047649f39f028e66b560a3acb8e"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:16.603255 14389 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.007s sys 0.000s
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:16.643175 14404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:16.644924 14389 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260808 10:41:16.645092 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "c5cb45edc3e2403c9239fbd9856d75f7"
format_stamp: "Formatted at 2026-08-08 10:41:16 on dist-test-slave-ss4n"
server_key: "c065796f4b3cc96cccf9fc45efd0a841"
server_key_iv: "f2433047649f39f028e66b560a3acb8e"
server_key_version: "kuduclusterkey@0"
I20260808 10:41:16.645336 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:41:16.671402 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:41:16.675774 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:41:16.676075 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:41:16.676878 14389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:41:16.678156 14389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:41:16.678277 14389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:16.678408 14389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:41:16.678500 14389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:41:16.694341 14389 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:36751
I20260808 10:41:16.695597 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:41:16.702893 14525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:36751 every 8 connection(s)
I20260808 10:41:16.703363 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 14389
I20260808 10:41:16.703502 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:41:16.703827 1801 external_mini_cluster.cc:1428] Setting key ea4f53456116e346e6d3d66fc5fa826b
Aug 08 10:41:16 dist-test-slave-ss4n krb5kdc[13496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185676, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:41:16.720583 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.705297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:55619 (local address 127.1.194.126:45393)
0808 10:41:16.705539 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.705543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.707370 (+ 1827us) server_negotiation.cc:407] Connection header received
0808 10:41:16.708585 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.708589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.708639 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.708684 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:41:16.710262 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.711060 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.712648 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.712983 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.713068 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.716101 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:41:16.716104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:41:16.716107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:41:16.716134 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:41:16.718189 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:16.718825 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:16.718829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:16.718830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:16.718912 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:16.719298 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:16.719302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:16.719303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:16.719529 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:41:16.719611 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.720289 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.720410 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260808 10:41:16.722921 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.705855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45393 (local address 127.1.194.67:55619)
0808 10:41:16.707218 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.707276 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.708336 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.709071 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.709080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.709435 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:41:16.710106 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.710123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.711210 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.711213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.712511 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.712521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.713268 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.713271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.713467 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.714193 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:41:16.714213 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:41:16.715898 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:41:16.718327 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:41:16.718333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:41:16.718351 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:41:16.718706 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:41:16.719020 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:41:16.719023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:41:16.719025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:41:16.719192 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:41:16.719679 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:41:16.719686 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:41:16.719995 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.721163 (+ 1168us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.721662 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1129,"thread_start_us":245,"threads_started":1}
I20260808 10:41:16.724331 14526 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45393
I20260808 10:41:16.724583 14526 heartbeater.cc:464] Registering TS with master...
I20260808 10:41:16.725116 14526 heartbeater.cc:511] Master 127.1.194.126:45393 requested a full tablet report, sending...
I20260808 10:41:16.726253 14009 ts_manager.cc:194] Registered new tserver with Master: c5cb45edc3e2403c9239fbd9856d75f7 (127.1.194.67:36751)
I20260808 10:41:16.726902 14009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:55619
I20260808 10:41:16.730360 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:41:16.744123 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.733981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.1.194.126:45393)
0808 10:41:16.734135 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.734162 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.734817 (+ 655us) server_negotiation.cc:407] Connection header received
0808 10:41:16.735012 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.735015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.735061 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.735108 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:41:16.736066 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.736824 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.737998 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.738302 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.738369 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.740148 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:41:16.740152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:41:16.740155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:41:16.740180 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:41:16.741918 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:16.742629 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:16.742632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:16.742634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:16.742719 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:41:16.743041 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:41:16.743044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:41:16.743046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:41:16.743268 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:41:16.743369 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.743797 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.743943 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 10:41:16.747713 14009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:41:16.751669 14009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:41:16.769138 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.761336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36751 (local address 127.0.0.1:48550)
0808 10:41:16.762911 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.762979 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.763105 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.763885 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.763893 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.763922 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:41:16.764084 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.764093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.765739 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.765743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.767233 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.767243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.768508 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.768513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.768649 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.768698 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.768793 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.768941 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1384,"mutex_wait_us":3,"thread_start_us":233,"threads_started":1}
I20260808 10:41:16.769634 14538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.762283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38249 (local address 127.0.0.1:43172)
0808 10:41:16.764088 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.764101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.764190 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.764757 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.764760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.764776 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:41:16.764858 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.764863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.766227 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.766230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.767540 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.767551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.768827 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.768830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.768926 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.768949 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.769034 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.769104 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"mutex_wait_us":66,"thread_start_us":94,"threads_started":1}
I20260808 10:41:16.769455 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.762460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.1.194.65:38249)
0808 10:41:16.762884 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.762894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.764280 (+ 1386us) server_negotiation.cc:407] Connection header received
0808 10:41:16.764355 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.764359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.764515 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.764636 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:41:16.764989 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.766085 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.767684 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.768508 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.768614 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.768761 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.769216 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.769316 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":107,"threads_started":1}
I20260808 10:41:16.770329 14539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.761532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.1.194.67:36751)
0808 10:41:16.763383 (+ 1851us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.763393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.763414 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:41:16.763489 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.763493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.763686 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.763841 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:41:16.764234 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.765591 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.767425 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.768364 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.769842 (+ 1478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.769995 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.770065 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.770177 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":128,"threads_started":1}
I20260808 10:41:16.770983 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.762319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.1.194.66:44893)
0808 10:41:16.763190 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.763201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.763441 (+ 240us) server_negotiation.cc:407] Connection header received
0808 10:41:16.763604 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.763608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.763755 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.763960 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:41:16.764370 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.766565 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.769177 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.770090 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.770173 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.770300 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.770735 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.770840 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":115,"threads_started":1}
I20260808 10:41:16.771384 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.761905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:44893 (local address 127.0.0.1:52962)
0808 10:41:16.763316 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.763328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.763470 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.764104 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.764107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.764128 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:41:16.764229 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.764235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.766719 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.766722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.768025 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.768036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.770417 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.770423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.770549 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.770578 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.771157 (+ 579us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.771246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":32,"thread_start_us":88,"threads_started":1}
I20260808 10:41:16.773459 14450 tablet_service.cc:1467] Processing CreateTablet for tablet d0a8d50d13154a7bb1937c40611e0e19 (DEFAULT_TABLE table=test-table [id=7039719216924f1a979ad21cb0578def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:41:16.774621 14450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0a8d50d13154a7bb1937c40611e0e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:41:16.774614 14164 tablet_service.cc:1467] Processing CreateTablet for tablet d0a8d50d13154a7bb1937c40611e0e19 (DEFAULT_TABLE table=test-table [id=7039719216924f1a979ad21cb0578def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:41:16.776064 14164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0a8d50d13154a7bb1937c40611e0e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:41:16.781258 14314 tablet_service.cc:1467] Processing CreateTablet for tablet d0a8d50d13154a7bb1937c40611e0e19 (DEFAULT_TABLE table=test-table [id=7039719216924f1a979ad21cb0578def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:41:16.782435 14314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0a8d50d13154a7bb1937c40611e0e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:41:16.784953 14544 tablet_bootstrap.cc:492] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Bootstrap starting.
I20260808 10:41:16.792940 14546 tablet_bootstrap.cc:492] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816: Bootstrap starting.
I20260808 10:41:16.794628 14544 tablet_bootstrap.cc:654] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Neither blocks nor log segments found. Creating new log.
I20260808 10:41:16.795909 14544 log.cc:824] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:41:16.796458 14546 tablet_bootstrap.cc:654] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816: Neither blocks nor log segments found. Creating new log.
I20260808 10:41:16.797260 14545 tablet_bootstrap.cc:492] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Bootstrap starting.
I20260808 10:41:16.799064 14546 log.cc:824] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816: Log is configured to *not* fsync() on all Append() calls
I20260808 10:41:16.800565 14545 tablet_bootstrap.cc:654] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Neither blocks nor log segments found. Creating new log.
I20260808 10:41:16.802369 14545 log.cc:824] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Log is configured to *not* fsync() on all Append() calls
I20260808 10:41:16.802835 14544 tablet_bootstrap.cc:492] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: No bootstrap required, opened a new log
I20260808 10:41:16.803057 14544 ts_tablet_manager.cc:1397] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260808 10:41:16.804704 14545 tablet_bootstrap.cc:492] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: No bootstrap required, opened a new log
I20260808 10:41:16.804878 14545 ts_tablet_manager.cc:1397] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260808 10:41:16.809113 14544 raft_consensus.cc:348] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.809208 14545 raft_consensus.cc:348] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.809425 14544 raft_consensus.cc:374] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:41:16.809481 14544 raft_consensus.cc:729] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5cb45edc3e2403c9239fbd9856d75f7, State: Initialized, Role: FOLLOWER
I20260808 10:41:16.809497 14545 raft_consensus.cc:374] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:41:16.809566 14545 raft_consensus.cc:729] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961967a608234c0e93c7ae363c762a75, State: Initialized, Role: FOLLOWER
I20260808 10:41:16.810007 14544 consensus_queue.cc:260] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.810345 14545 consensus_queue.cc:260] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.811246 14544 ts_tablet_manager.cc:1428] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 10:41:16.811386 14545 ts_tablet_manager.cc:1428] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:41:16.811642 14526 heartbeater.cc:503] Master 127.1.194.126:45393 was elected leader, sending a full tablet report...
I20260808 10:41:16.811793 14384 heartbeater.cc:503] Master 127.1.194.126:45393 was elected leader, sending a full tablet report...
I20260808 10:41:16.813735 14546 tablet_bootstrap.cc:492] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816: No bootstrap required, opened a new log
I20260808 10:41:16.813956 14546 ts_tablet_manager.cc:1397] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.000s
I20260808 10:41:16.820366 14546 raft_consensus.cc:348] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.820675 14546 raft_consensus.cc:374] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:41:16.820736 14546 raft_consensus.cc:729] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5141a87c81f44392802a66b59c42d816, State: Initialized, Role: FOLLOWER
I20260808 10:41:16.821230 14546 consensus_queue.cc:260] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.822095 14237 heartbeater.cc:503] Master 127.1.194.126:45393 was elected leader, sending a full tablet report...
I20260808 10:41:16.822551 14546 ts_tablet_manager.cc:1428] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
W20260808 10:41:16.823585 14385 tablet.cc:2369] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:41:16.834476 14552 raft_consensus.cc:482] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:41:16.834714 14552 raft_consensus.cc:504] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.835889 14552 leader_election.cc:288] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5cb45edc3e2403c9239fbd9856d75f7 (127.1.194.67:36751), 961967a608234c0e93c7ae363c762a75 (127.1.194.66:44893)
I20260808 10:41:16.841521 14539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.836259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:41303 (local address 127.1.194.67:36751)
0808 10:41:16.836391 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.836396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.836719 (+ 323us) server_negotiation.cc:407] Connection header received
0808 10:41:16.836975 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.836978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.837035 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.837099 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:41:16.837696 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.838473 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.840140 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.840768 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.840869 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.840923 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.841274 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.841349 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:41:16.842010 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.836109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:36751 (local address 127.1.194.65:41303)
0808 10:41:16.836602 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.836619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.836829 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.837260 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.837262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.837279 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:41:16.837535 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.837547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.838714 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.838717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.839989 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.840000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.841023 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.841027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.841139 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.841161 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.841730 (+ 569us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.841817 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":186,"threads_started":1}
I20260808 10:41:16.842649 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.837161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:34249 (local address 127.1.194.66:44893)
0808 10:41:16.837278 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.837283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.837841 (+ 558us) server_negotiation.cc:407] Connection header received
0808 10:41:16.838022 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.838025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.838074 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.838123 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:41:16.838506 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.839305 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.840881 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.841539 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.841646 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.841696 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.842437 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.842522 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:41:16.842713 14455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0a8d50d13154a7bb1937c40611e0e19" candidate_uuid: "5141a87c81f44392802a66b59c42d816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" is_pre_election: true
I20260808 10:41:16.843060 14455 raft_consensus.cc:2454] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5141a87c81f44392802a66b59c42d816 in term 0.
I20260808 10:41:16.843287 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.836780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:44893 (local address 127.1.194.65:34249)
0808 10:41:16.837614 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.837630 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:41:16.837908 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:41:16.838244 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:41:16.838247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:41:16.838261 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:41:16.838384 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.838389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.839511 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.839514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.840737 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:41:16.840748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.841978 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.841984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:41:16.842292 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.842315 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:41:16.843000 (+ 685us) client_negotiation.cc:240] Negotiation successful
0808 10:41:16.843071 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":109,"threads_started":1}
I20260808 10:41:16.843675 14119 leader_election.cc:302] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5141a87c81f44392802a66b59c42d816, c5cb45edc3e2403c9239fbd9856d75f7; no voters:
I20260808 10:41:16.844013 14552 raft_consensus.cc:2781] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:41:16.844108 14552 raft_consensus.cc:482] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:41:16.844189 14552 raft_consensus.cc:3037] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:41:16.844231 14335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0a8d50d13154a7bb1937c40611e0e19" candidate_uuid: "5141a87c81f44392802a66b59c42d816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961967a608234c0e93c7ae363c762a75" is_pre_election: true
I20260808 10:41:16.844537 14335 raft_consensus.cc:2454] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5141a87c81f44392802a66b59c42d816 in term 0.
I20260808 10:41:16.845770 14552 raft_consensus.cc:504] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.846179 14552 leader_election.cc:288] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [CANDIDATE]: Term 1 election: Requested vote from peers c5cb45edc3e2403c9239fbd9856d75f7 (127.1.194.67:36751), 961967a608234c0e93c7ae363c762a75 (127.1.194.66:44893)
I20260808 10:41:16.846603 14335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0a8d50d13154a7bb1937c40611e0e19" candidate_uuid: "5141a87c81f44392802a66b59c42d816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961967a608234c0e93c7ae363c762a75"
I20260808 10:41:16.846735 14335 raft_consensus.cc:3037] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:41:16.848197 14455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0a8d50d13154a7bb1937c40611e0e19" candidate_uuid: "5141a87c81f44392802a66b59c42d816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5cb45edc3e2403c9239fbd9856d75f7"
I20260808 10:41:16.848402 14455 raft_consensus.cc:3037] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:41:16.848799 14335 raft_consensus.cc:2454] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5141a87c81f44392802a66b59c42d816 in term 1.
I20260808 10:41:16.849272 14120 leader_election.cc:302] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5141a87c81f44392802a66b59c42d816, 961967a608234c0e93c7ae363c762a75; no voters:
I20260808 10:41:16.849478 14552 raft_consensus.cc:2781] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:41:16.849766 14552 raft_consensus.cc:686] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [term 1 LEADER]: Becoming Leader. State: Replica: 5141a87c81f44392802a66b59c42d816, State: Running, Role: LEADER
I20260808 10:41:16.850149 14552 consensus_queue.cc:237] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } }
I20260808 10:41:16.850469 14455 raft_consensus.cc:2454] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5141a87c81f44392802a66b59c42d816 in term 1.
I20260808 10:41:16.854130 14007 catalog_manager.cc:5495] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5141a87c81f44392802a66b59c42d816 (127.1.194.65). New cstate: current_term: 1 leader_uuid: "5141a87c81f44392802a66b59c42d816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5141a87c81f44392802a66b59c42d816" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38249 } health_report { overall_health: HEALTHY } } }
I20260808 10:41:16.875631 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.871904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.1.194.65:38249)
0808 10:41:16.872032 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:41:16.872039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:41:16.872057 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:41:16.872266 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:41:16.872269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:41:16.872326 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:41:16.872439 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:41:16.872699 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.873213 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.874166 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:41:16.874387 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:41:16.874494 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:41:16.874819 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:41:16.875224 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:41:16.875357 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:41:16.875428 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:41:16.875485 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:41:16.882195 14335 raft_consensus.cc:1261] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 1 FOLLOWER]: Refusing update from remote peer 5141a87c81f44392802a66b59c42d816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:41:16.882195 14455 raft_consensus.cc:1261] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 1 FOLLOWER]: Refusing update from remote peer 5141a87c81f44392802a66b59c42d816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:41:16.882907 14555 consensus_queue.cc:1035] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "961967a608234c0e93c7ae363c762a75" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:41:16.883234 14552 consensus_queue.cc:1035] T d0a8d50d13154a7bb1937c40611e0e19 P 5141a87c81f44392802a66b59c42d816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5cb45edc3e2403c9239fbd9856d75f7" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:41:16.891040 14558 mvcc.cc:205] Tried to move back new op lower bound from 7316216532502036480 to 7316216532382052352. Current Snapshot: MvccSnapshot[applied={T|T < 7316216532502036480}]
I20260808 10:41:16.895083 14560 mvcc.cc:205] Tried to move back new op lower bound from 7316216532502036480 to 7316216532382052352. Current Snapshot: MvccSnapshot[applied={T|T < 7316216532502036480}]
I20260808 10:41:16.896508 14561 mvcc.cc:205] Tried to move back new op lower bound from 7316216532502036480 to 7316216532382052352. Current Snapshot: MvccSnapshot[applied={T|T < 7316216532502036480}]
I20260808 10:41:16.909858 14007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:41:16.910121 14007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:41:16.915274 14007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 448e71402c4449c892e8a7401215b3ca: Sending DeleteTablet for 3 replicas of tablet d0a8d50d13154a7bb1937c40611e0e19
I20260808 10:41:16.916270 14450 tablet_service.cc:1514] Processing DeleteTablet for tablet d0a8d50d13154a7bb1937c40611e0e19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:41:16 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:48550
I20260808 10:41:16.917572 14314 tablet_service.cc:1514] Processing DeleteTablet for tablet d0a8d50d13154a7bb1937c40611e0e19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:41:16 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:52962
I20260808 10:41:16.919126 14164 tablet_service.cc:1514] Processing DeleteTablet for tablet d0a8d50d13154a7bb1937c40611e0e19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:41:16 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:43172
I20260808 10:41:16.920322 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 14098
I20260808 10:41:16.923601 14569 tablet_replica.cc:330] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: stopping tablet replica
I20260808 10:41:16.924078 14569 raft_consensus.cc:2229] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:41:16.924732 14569 raft_consensus.cc:2258] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:41:16.926864 14569 ts_tablet_manager.cc:1905] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:41:16.930716 14569 ts_tablet_manager.cc:1918] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:41:16.930894 14569 log.cc:1196] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/d0a8d50d13154a7bb1937c40611e0e19
I20260808 10:41:16.931317 14569 ts_tablet_manager.cc:1939] T d0a8d50d13154a7bb1937c40611e0e19 P 961967a608234c0e93c7ae363c762a75: Deleting consensus metadata
I20260808 10:41:16.931381 14568 tablet_replica.cc:330] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: stopping tablet replica
I20260808 10:41:16.931844 14568 raft_consensus.cc:2229] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:41:16.932057 13996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:38249 (error 108)
I20260808 10:41:16.932206 14568 raft_consensus.cc:2258] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:41:16.933030 13996 catalog_manager.cc:4841] TS 961967a608234c0e93c7ae363c762a75 (127.1.194.66:44893): tablet d0a8d50d13154a7bb1937c40611e0e19 (table test-table [id=7039719216924f1a979ad21cb0578def]) successfully deleted
I20260808 10:41:16.933823 14568 ts_tablet_manager.cc:1905] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:41:16.934602 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:41:16.934322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38249 (local address 127.0.0.1:43202)
0808 10:41:16.934434 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 10:41:16.934515 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:38249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:41:16.934803 13996 catalog_manager.cc:4584] TS 5141a87c81f44392802a66b59c42d816 (127.1.194.65:38249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0a8d50d13154a7bb1937c40611e0e19: Network error: Client connection negotiation failed: client connection to 127.1.194.65:38249: connect: Connection refused (error 111)
I20260808 10:41:16.935302 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 14245
I20260808 10:41:16.937955 14568 ts_tablet_manager.cc:1918] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:41:16.938077 14568 log.cc:1196] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/d0a8d50d13154a7bb1937c40611e0e19
I20260808 10:41:16.938490 14568 ts_tablet_manager.cc:1939] T d0a8d50d13154a7bb1937c40611e0e19 P c5cb45edc3e2403c9239fbd9856d75f7: Deleting consensus metadata
I20260808 10:41:16.940086 13995 catalog_manager.cc:4841] TS c5cb45edc3e2403c9239fbd9856d75f7 (127.1.194.67:36751): tablet d0a8d50d13154a7bb1937c40611e0e19 (table test-table [id=7039719216924f1a979ad21cb0578def]) successfully deleted
I20260808 10:41:16.946796 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 14389
I20260808 10:41:16.955080 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 13977
I20260808 10:41:16.963351 1801 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:41:17.308411 1801 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:41:17.308605 1801 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:41:17.653771 1801 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:41:17.654 UTC [13529] LOG: received smart shutdown request
2026-08-08 10:41:17.658 UTC [13529] LOG: background worker "logical replication launcher" (PID 13538) exited with exit code 1
2026-08-08 10:41:17.658 UTC [13533] LOG: shutting down
2026-08-08 10:41:17.857 UTC [13529] LOG: database system is shut down
2026-08-08T10:41:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50274 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:41:18 dist-test-slave-ss4n krb5kdc[14580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:41:18 dist-test-slave-ss4n krb5kdc[14580](info): set up 2 sockets
Aug 08 10:41:18 dist-test-slave-ss4n krb5kdc[14580](info): commencing operation
krb5kdc: starting...
W20260808 10:41:19.081096 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:41:19 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:41:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:41:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/rangeradmin_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/rangeradmin_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/rangerlookup_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/rangerlookup_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
I20260808 10:41:19.280197 1801 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBjlYe1/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/postgres -l logfile start
2026-08-08 10:41:22.058 UTC [14613] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:41:22.058 UTC [14613] LOG: listening on IPv4 address "127.1.194.84", port 53267
2026-08-08 10:41:22.062 UTC [14613] LOG: listening on Unix socket "/tmp/.s.PGSQL.53267"
2026-08-08 10:41:22.083 UTC [14616] LOG: database system was shut down at 2026-08-08 10:41:20 UTC
2026-08-08 10:41:22.087 UTC [14613] LOG: database system is ready to accept connections
I20260808 10:41:23.098551 1801 mini_postgres.cc:96] Postgres bound to 53267
2026-08-08 10:41:23.106 UTC [14625] FATAL: database "slave" does not exist
127.1.194.84:53267 - accepting connections
I20260808 10:41:23.107525 1801 mini_ranger.cc:161] Starting Ranger...
I20260808 10:41:23.125530 1801 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:41:23.671197 1801 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:41:23.671304 1801 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-admin
2026-08-08 10:41:23,958 [I] DB FLAVOR :POSTGRES
2026-08-08 10:41:23,958 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:41:23,958 [I] Checking connection..
2026-08-08 10:41:23,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:41:24,219 [I] Checking connection passed.
2026-08-08 10:41:24,220 [I] --------- Verifying version history table ---------
2026-08-08 10:41:24,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:41:24,471 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:41:24,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:41:24,753 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:41:24,753 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:41:24,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:41:25.039 UTC [14692] WARNING: there is no transaction in progress
2026-08-08 10:41:25,047 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:41:25,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:41:25,309 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:41:25,309 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:41:25,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:41:25,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:41:25,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-08 10:41:26,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:41:26,293 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:41:26,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:41:26,553 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:41:26,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:41:26,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:41:27,123 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:41:27,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:41:28.172 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.189 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.207 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.223 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.241 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.262 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.332 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.340 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.357 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.369 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.381 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.397 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.408 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.416 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.425 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:28.433 UTC [14807] WARNING: there is no transaction in progress
2026-08-08 10:41:29,168 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:41:29,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:41:29,431 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:41:29,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:41:29,710 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:41:29,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:41:29,987 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:41:29,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:41:30,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:41:30,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-08 10:41:30,744 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:41:30,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:41:31,011 [I] DB_PATCHES have already been applied
I20260808 10:41:31.017879 1801 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBjlYe1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:41:31.017947 1801 mini_ranger.cc:193] Using host: 127.1.194.84
I20260808 10:41:31.022217 1801 mini_ranger.cc:227] Ranger admin URL: http://127.1.194.84:35789
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:41:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:41:32.170356 1801 mini_ranger.cc:160] Time spent starting Ranger: real 9.063s user 0.002s sys 0.010s
Aug 08, 2026 10:41:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:41:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 10:41:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:41:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:41:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:41:55.000487 1801 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.194.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.194.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/rangerkms_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.194.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/rangerkms_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.194.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.194.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.194.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/HTTP_127.1.194.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:41:55 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:41:55.107321 1801 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:41:55.119174 1801 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:41:56.128063 1801 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:41:56.128155 1801 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms
I20260808 10:41:56.128175 1801 mini_ranger_kms.cc:227] Using postgres at 127.1.194.84:53267
2026-08-08 10:41:56,309 [I] DB FLAVOR :POSTGRES
2026-08-08 10:41:56,309 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:41:56,309 [I] Checking connection
2026-08-08 10:41:56,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:41:56,574 [I] connection success
2026-08-08 10:41:56,574 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:41:56,574 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:41:56,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:41:56,817 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:41:56,818 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:41:56,818 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:41:56,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.194.84:53267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:41:57,095 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:41:57.575019 1801 mini_ranger_kms.cc:310] Created kms service
I20260808 10:41:57.805173 1801 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:41:57.860882 1801 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:41:57 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:41:58.318427 1801 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:41:58.318626 1801 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBjlYe1/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBjlYe1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:41:58.318650 1801 mini_ranger_kms.cc:240] Using host: 127.1.194.84
I20260808 10:41:58.323284 1801 mini_ranger_kms.cc:276] Ranger KMS PID: 14998
I20260808 10:41:58.323375 1801 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.194.84:38553
Aug 08, 2026 10:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:41:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 10:42:00 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.194.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:00.776623 1801 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.669s user 0.020s sys 0.138s
I20260808 10:42:00.776782 1801 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:42:00.776917 1801 mini_ranger_kms.cc:401] 127.1.194.84:38553/kms/v1/keys
Aug 08 10:42:01 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.194.84@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
Aug 08 10:42:01 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.194.84@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
Aug 08 10:42:04 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:42:04.109812 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34321
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34321
--ranger_config_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:04.222545 15052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:04.222873 15052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:04.222959 15052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:04.227632 15052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:04.227725 15052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:04.227762 15052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:04.227800 15052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:04.227835 15052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:04.233400 15052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.194.84:38553/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34321
--ranger_config_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:04.234498 15052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:04.235543 15052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:04.245150 15057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:04.245187 15058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:04.245240 15060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:04.245419 15052 server_base.cc:1034] running on GCE node
I20260808 10:42:04.246254 15052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:04.247656 15052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:04.248870 15052 hybrid_clock.cc:648] HybridClock initialized: now 1786185724248857 us; error 60 us; skew 500 ppm
Aug 08 10:42:04 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185724, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:04.252573 15052 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:04.253974 15052 webserver.cc:460] Webserver started at http://127.1.194.126:39315/ using document root <none> and password file <none>
I20260808 10:42:04.254721 15052 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:04.254837 15052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:04.255100 15052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:42:04 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185724, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:04.302729 15052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "590ad46526064e858637a890549e3566"
format_stamp: "Formatted at 2026-08-08 10:42:04 on dist-test-slave-ss4n"
server_key: "ef35ed5da050213a0fb2ae2480be7ca4"
server_key_iv: "573a405c75ba35269bc4a5df2ccf078f"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:04.303371 15052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "590ad46526064e858637a890549e3566"
format_stamp: "Formatted at 2026-08-08 10:42:04 on dist-test-slave-ss4n"
server_key: "ef35ed5da050213a0fb2ae2480be7ca4"
server_key_iv: "573a405c75ba35269bc4a5df2ccf078f"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:04.307376 15052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:42:04.321904 15067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:04.323318 15052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:04.323482 15052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "590ad46526064e858637a890549e3566"
format_stamp: "Formatted at 2026-08-08 10:42:04 on dist-test-slave-ss4n"
server_key: "ef35ed5da050213a0fb2ae2480be7ca4"
server_key_iv: "573a405c75ba35269bc4a5df2ccf078f"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:04.323640 15052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:04.351416 15052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:04.354908 15052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:04.355141 15052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:04.364462 15052 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34321
I20260808 10:42:04.364471 15123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34321 every 8 connection(s)
I20260808 10:42:04.365602 15052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:04.368197 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15052
I20260808 10:42:04.368323 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:42:04.368654 1801 external_mini_cluster.cc:1428] Setting key c51fc7778a7a0b102598840eaa94568e
I20260808 10:42:04.368669 15124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:04.376794 15124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Bootstrap starting.
I20260808 10:42:04.379606 15124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Neither blocks nor log segments found. Creating new log.
Aug 08 10:42:04 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:04.380682 15124 log.cc:824] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:04.383163 15124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: No bootstrap required, opened a new log
I20260808 10:42:04.387219 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:04.370227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.1.194.126:34321)
0808 10:42:04.370714 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:04.370735 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:04.370778 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:42:04.371865 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:04.371891 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:04.372341 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:04.372830 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:04.373886 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:04.375561 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:04.376783 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:04.377530 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:04.377698 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:04.381006 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:04.381012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:04.381030 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:04.381063 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:04.383953 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:04.384522 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:04.384527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:04.384533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:04.384665 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:04.384986 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:04.384989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:04.384991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:04.385517 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:04.385716 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:04.386029 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 10:42:04.386310 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":141,"threads_started":1}
I20260808 10:42:04.387265 15124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:04.387639 15124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:04.387727 15124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 590ad46526064e858637a890549e3566, State: Initialized, Role: FOLLOWER
I20260808 10:42:04.388278 15124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:04.388489 15124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:04.388566 15124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:04.388672 15124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:04.390008 15124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:04.390421 15124 leader_election.cc:302] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 590ad46526064e858637a890549e3566; no voters:
I20260808 10:42:04.390799 15124 leader_election.cc:288] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:42:04.391230 15130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:04.392009 15130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 LEADER]: Becoming Leader. State: Replica: 590ad46526064e858637a890549e3566, State: Running, Role: LEADER
I20260808 10:42:04.392266 15124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:04.392398 15130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:04.394196 15132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 590ad46526064e858637a890549e3566. Latest consensus state: current_term: 1 leader_uuid: "590ad46526064e858637a890549e3566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } } }
I20260808 10:42:04.394321 15132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:04.394645 15131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "590ad46526064e858637a890549e3566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } } }
I20260808 10:42:04.394721 15131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:04.394728 15144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:04.394946 15124 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:42:04.401361 15144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:04.410880 15144 catalog_manager.cc:1347] Generated new cluster ID: 7017de4031d34b9d9de24c4585d60010
I20260808 10:42:04.411022 15144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:04.465857 15144 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:42:04.467207 15144 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:04.481557 15144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Generated new TSK 0
I20260808 10:42:04.483038 15144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:42:05.249611 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34321
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:05.403975 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:05.404320 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:05.404387 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:05.410501 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:05.410588 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:05.410678 15171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:05.417776 15171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.194.84:38553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:05.418962 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:05.420094 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:05.432489 15178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:05.432904 15181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:05.433099 15179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:05.433547 15171 server_base.cc:1034] running on GCE node
I20260808 10:42:05.434038 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:05.434902 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:05.436106 15171 hybrid_clock.cc:648] HybridClock initialized: now 1786185725436084 us; error 36 us; skew 500 ppm
Aug 08 10:42:05 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:05.439790 15171 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:05.440778 15171 webserver.cc:460] Webserver started at http://127.1.194.65:41425/ using document root <none> and password file <none>
I20260808 10:42:05.441344 15171 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:05.441404 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:05.441669 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:42:05 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:05.472506 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "52945ce182ce4fd0a998f9db91dfc9d0"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "0e95a2a0cc6b85b3278ec53be6aae0dd"
server_key_iv: "4fb9a7169442c51d22aaaf57a34fb503"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:05.475277 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "52945ce182ce4fd0a998f9db91dfc9d0"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "0e95a2a0cc6b85b3278ec53be6aae0dd"
server_key_iv: "4fb9a7169442c51d22aaaf57a34fb503"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:05.487059 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.004s
I20260808 10:42:05.506570 15191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:05.508366 15171 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 10:42:05.508522 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "52945ce182ce4fd0a998f9db91dfc9d0"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "0e95a2a0cc6b85b3278ec53be6aae0dd"
server_key_iv: "4fb9a7169442c51d22aaaf57a34fb503"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:05.508687 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:05.551295 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:05.555236 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:05.555540 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:05.556310 15171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:05.557518 15171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:05.557631 15171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 10:42:05.557765 15171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:05.557864 15171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:05.569355 15171 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:45829
I20260808 10:42:05.570792 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:05.571159 15310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:45829 every 8 connection(s)
I20260808 10:42:05.577514 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15171
I20260808 10:42:05.577678 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:42:05.578045 1801 external_mini_cluster.cc:1428] Setting key 24bf888ae641af990da4ef11cc80caf7
Aug 08 10:42:05 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:05.590978 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:05.574449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34321 (local address 127.1.194.65:55897)
0808 10:42:05.576093 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:05.576152 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:05.577227 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:05.578313 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:05.578321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:05.578877 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:05.579508 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:05.579526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:05.580658 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:05.580662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:05.582021 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:05.582032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:05.582820 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:05.582823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:05.583013 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:05.583728 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:05.583749 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:05.585528 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:05.587742 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:05.587748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:05.587764 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:05.588085 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:05.588391 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:05.588394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:05.588397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:05.588562 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:05.589031 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:05.589038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:05.589370 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 10:42:05.589596 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 10:42:05.589933 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"thread_start_us":138,"threads_started":1}
I20260808 10:42:05.591739 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:05.573814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:55897 (local address 127.1.194.126:34321)
0808 10:42:05.577800 (+ 3986us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:05.577805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:05.577822 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:42:05.577868 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:05.577871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:05.577920 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:05.578005 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:05.579681 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:05.580502 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:05.582182 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:05.582532 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:05.582646 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:05.585730 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:05.585734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:05.585736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:05.585761 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:05.587588 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:05.588202 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:05.588205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:05.588207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:05.588295 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:05.588679 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:05.588682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:05.588684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:05.588892 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:05.588980 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:05.591422 (+ 2442us) server_negotiation.cc:299] Negotiation successful
0808 10:42:05.591558 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3857,"thread_start_us":102,"threads_started":1}
I20260808 10:42:05.593592 15311 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34321
I20260808 10:42:05.593842 15311 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:05.594373 15311 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:05.596129 15085 ts_manager.cc:194] Registered new tserver with Master: 52945ce182ce4fd0a998f9db91dfc9d0 (127.1.194.65:45829)
I20260808 10:42:05.597635 15085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:55897
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Aug 08 10:42:05 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:42:05.645293 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34321
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:05.813324 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:05.813835 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:05.813966 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:05.820569 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:05.820719 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:05.820863 15317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:05.829177 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.194.84:38553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:05.830881 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:05.832315 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:05.845855 15324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:05.845866 15327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:05.846562 15323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:05.847407 15317 server_base.cc:1034] running on GCE node
I20260808 10:42:05.848058 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:05.848891 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:05.852190 15317 hybrid_clock.cc:648] HybridClock initialized: now 1786185725852078 us; error 120 us; skew 500 ppm
Aug 08 10:42:05 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:05.855835 15317 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:05.857113 15317 webserver.cc:460] Webserver started at http://127.1.194.66:38529/ using document root <none> and password file <none>
I20260808 10:42:05.857869 15317 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:05.857961 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:05.858222 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:42:05 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:05.887044 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "8c374452862547feb2ced49786cf479a"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "8236dea9535a48f0de2f74f52c7cdbfa"
server_key_iv: "6447562822e163e29c87bd077a566d26"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:05.894047 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "8c374452862547feb2ced49786cf479a"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "8236dea9535a48f0de2f74f52c7cdbfa"
server_key_iv: "6447562822e163e29c87bd077a566d26"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:05.922569 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.028s user 0.004s sys 0.000s
I20260808 10:42:05.940383 15334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:05.942406 15317 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:42:05.942595 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "8c374452862547feb2ced49786cf479a"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "8236dea9535a48f0de2f74f52c7cdbfa"
server_key_iv: "6447562822e163e29c87bd077a566d26"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:05.942777 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:05.975804 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:05.980165 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:05.980537 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:05.981343 15317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:05.982527 15317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:05.982648 15317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:05.982752 15317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:05.982818 15317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:05.995620 15317 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:44705
I20260808 10:42:05.998207 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:06.000720 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15317
I20260808 10:42:06.001106 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:42:06.001464 1801 external_mini_cluster.cc:1428] Setting key a81cf483797062daf4055edf0656f1d0
I20260808 10:42:06.002249 15453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:44705 every 8 connection(s)
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:42:06.033776 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:06.008699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:53275 (local address 127.1.194.126:34321)
0808 10:42:06.008827 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:06.008832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:06.010268 (+ 1436us) server_negotiation.cc:407] Connection header received
0808 10:42:06.012931 (+ 2663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:06.012935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:06.013000 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:06.013063 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:06.019345 (+ 6282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.020224 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.022139 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.022534 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.022619 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:06.027439 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:06.027443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:06.027446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:06.027491 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:06.029792 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:06.031814 (+ 2022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:06.031818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:06.031820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:06.031914 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:06.032360 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:06.032363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:06.032365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:06.032621 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:06.032714 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:06.033419 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 10:42:06.033599 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:42:06.035804 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:06.007573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34321 (local address 127.1.194.66:53275)
0808 10:42:06.010097 (+ 2524us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:06.010159 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:06.011287 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:06.013569 (+ 2282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:06.013580 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:06.013959 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:06.016127 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:06.016151 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.020491 (+ 4340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.020495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:06.021985 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:06.021997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.022844 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.022847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:06.023061 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:06.023887 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:06.023909 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:06.027213 (+ 3304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:06.029939 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:06.029945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:06.029963 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:06.030324 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:06.032036 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:06.032039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:06.032041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:06.032230 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:06.032786 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:06.032792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:06.033118 (+ 326us) client_negotiation.cc:769] Sending connection context
0808 10:42:06.034286 (+ 1168us) client_negotiation.cc:240] Negotiation successful
0808 10:42:06.034662 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2292,"thread_start_us":155,"threads_started":1}
I20260808 10:42:06.038126 15454 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34321
I20260808 10:42:06.038394 15454 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:06.039280 15454 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:42:06.042343 15085 ts_manager.cc:194] Registered new tserver with Master: 8c374452862547feb2ced49786cf479a (127.1.194.66:44705)
I20260808 10:42:06.043143 15085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:53275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:42:06.067981 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34321
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185725, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
W20260808 10:42:06.247689 15459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:06.248169 15459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:06.248307 15459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:06.255020 15459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:06.255190 15459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:06.255349 15459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:06.264719 15459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.194.84:38553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:06.266620 15459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:06.268121 15459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:06.288698 15464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:06.288689 15465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:06.288702 15467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:06.289306 15459 server_base.cc:1034] running on GCE node
I20260808 10:42:06.289873 15459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:06.290745 15459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:06.292047 15459 hybrid_clock.cc:648] HybridClock initialized: now 1786185726291972 us; error 93 us; skew 500 ppm
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:06.296422 15459 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:06.297811 15459 webserver.cc:460] Webserver started at http://127.1.194.67:33331/ using document root <none> and password file <none>
I20260808 10:42:06.298638 15459 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:06.298799 15459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:06.299106 15459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:06.335202 15459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "3321f97862384162bb1e207fa6862f5c"
format_stamp: "Formatted at 2026-08-08 10:42:06 on dist-test-slave-ss4n"
server_key: "2d7d5c0037692397a3d2af9c44dc7235"
server_key_iv: "8fd5e02467dcbeba9f7ceba071d5c834"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:06.336205 15459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "3321f97862384162bb1e207fa6862f5c"
format_stamp: "Formatted at 2026-08-08 10:42:06 on dist-test-slave-ss4n"
server_key: "2d7d5c0037692397a3d2af9c44dc7235"
server_key_iv: "8fd5e02467dcbeba9f7ceba071d5c834"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:06.351413 15459 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.004s sys 0.000s
I20260808 10:42:06.377679 15476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:06.379550 15459 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 10:42:06.379774 15459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "3321f97862384162bb1e207fa6862f5c"
format_stamp: "Formatted at 2026-08-08 10:42:06 on dist-test-slave-ss4n"
server_key: "2d7d5c0037692397a3d2af9c44dc7235"
server_key_iv: "8fd5e02467dcbeba9f7ceba071d5c834"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:06.380012 15459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:06.409102 15459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:06.412968 15459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:06.413228 15459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:06.414029 15459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:06.415174 15459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:06.415277 15459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:06.415366 15459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:06.415419 15459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:06.426285 15459 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:45529
I20260808 10:42:06.427577 15459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:06.436496 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15459
I20260808 10:42:06.436646 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:42:06.436959 1801 external_mini_cluster.cc:1428] Setting key 0757762a1d4309bd89f885b66ef6581f
I20260808 10:42:06.437203 15596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:45529 every 8 connection(s)
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:06.456063 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:06.439279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:38863 (local address 127.1.194.126:34321)
0808 10:42:06.439403 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:06.439408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:06.441039 (+ 1631us) server_negotiation.cc:407] Connection header received
0808 10:42:06.442365 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:06.442370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:06.442426 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:06.442482 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:06.444476 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.445298 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.447369 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.447711 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.447800 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:06.451311 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:06.451316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:06.451319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:06.451349 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:06.453317 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:06.454052 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:06.454055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:06.454057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:06.454154 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:06.454606 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:06.454610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:06.454612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:06.454863 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:06.454951 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:06.455719 (+ 768us) server_negotiation.cc:299] Negotiation successful
0808 10:42:06.455872 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:42:06.458076 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:06.439637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34321 (local address 127.1.194.67:38863)
0808 10:42:06.440863 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:06.440923 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:06.442084 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:06.442921 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:06.442930 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:06.443342 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:06.444291 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:06.444313 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.445469 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.445472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:06.447219 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:06.447230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:06.448072 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:06.448077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:06.448279 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:06.449037 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:06.449059 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:06.451090 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:06.453465 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:06.453472 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:06.453517 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:06.453909 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:06.454272 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:06.454275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:06.454278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:06.454482 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:06.455039 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:06.455047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:06.455405 (+ 358us) client_negotiation.cc:769] Sending connection context
0808 10:42:06.456515 (+ 1110us) client_negotiation.cc:240] Negotiation successful
0808 10:42:06.456894 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":156,"threads_started":1}
I20260808 10:42:06.459723 15597 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34321
I20260808 10:42:06.459971 15597 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:06.460572 15597 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:06.462062 15085 ts_manager.cc:194] Registered new tserver with Master: 3321f97862384162bb1e207fa6862f5c (127.1.194.67:45529)
I20260808 10:42:06.462910 15085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:38863
I20260808 10:42:06.464129 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:42:06.464946 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15171
I20260808 10:42:06.491415 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15317
I20260808 10:42:06.501869 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15459
I20260808 10:42:06.511622 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15052
I20260808 10:42:06.520009 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34321
--webserver_interface=127.1.194.126
--webserver_port=39315
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34321
--ranger_config_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:06.631944 15601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:06.632223 15601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:06.632263 15601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:06.636389 15601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:06.636447 15601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:06.636466 15601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:06.636483 15601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:06.636499 15601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:06.641948 15601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34321
--ranger_config_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:06.643021 15601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:06.644068 15601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:06.652699 15607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:06.652699 15606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:06.652689 15609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:06.653512 15601 server_base.cc:1034] running on GCE node
I20260808 10:42:06.654181 15601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:06.655597 15601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:06.656797 15601 hybrid_clock.cc:648] HybridClock initialized: now 1786185726656778 us; error 38 us; skew 500 ppm
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:06.660212 15601 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:06.661260 15601 webserver.cc:460] Webserver started at http://127.1.194.126:39315/ using document root <none> and password file <none>
I20260808 10:42:06.661935 15601 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:06.662026 15601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:06.746111 15601 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:42:06.749003 15618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:06.750674 15601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:06.750819 15601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "590ad46526064e858637a890549e3566"
format_stamp: "Formatted at 2026-08-08 10:42:04 on dist-test-slave-ss4n"
server_key: "ef35ed5da050213a0fb2ae2480be7ca4"
server_key_iv: "573a405c75ba35269bc4a5df2ccf078f"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:06.751547 15601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:06.777392 15601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:06.780906 15601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:06.781124 15601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:06.790120 15672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34321 every 8 connection(s)
I20260808 10:42:06.790096 15601 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34321
I20260808 10:42:06.791577 15601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:06.795915 15673 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:42:06.797593 15673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Bootstrap starting.
I20260808 10:42:06.799769 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15601
I20260808 10:42:06.800256 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:45829
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:34321
--webserver_port=41425
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:06.815251 15673 log.cc:824] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:06.822353 15673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:06.822784 15673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Bootstrap complete.
I20260808 10:42:06.827136 15673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:06.827548 15673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 590ad46526064e858637a890549e3566, State: Initialized, Role: FOLLOWER
I20260808 10:42:06.828089 15673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:06.828284 15673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:06.828378 15673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:06.828511 15673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:42:06.830478 15673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:06.830818 15673 leader_election.cc:302] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 590ad46526064e858637a890549e3566; no voters:
I20260808 10:42:06.831151 15673 leader_election.cc:288] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:42:06.831238 15677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:42:06.831564 15677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [term 2 LEADER]: Becoming Leader. State: Replica: 590ad46526064e858637a890549e3566, State: Running, Role: LEADER
I20260808 10:42:06.831909 15677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } }
I20260808 10:42:06.832499 15673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:06.832887 15677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 590ad46526064e858637a890549e3566. Latest consensus state: current_term: 2 leader_uuid: "590ad46526064e858637a890549e3566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } } }
I20260808 10:42:06.833240 15677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:06.833529 15677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "590ad46526064e858637a890549e3566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ad46526064e858637a890549e3566" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34321 } } }
I20260808 10:42:06.833698 15677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:06.834002 15684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:06.837432 15684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:06.837982 15673 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:42:06.838263 15684 catalog_manager.cc:1259] Loaded cluster ID: 7017de4031d34b9d9de24c4585d60010
I20260808 10:42:06.838356 15684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:06.840660 15684 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:06.841349 15684 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Loaded TSK: 0
I20260808 10:42:06.842620 15684 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:42:06.961172 15675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:06.962007 15675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:06.962137 15675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:06.968444 15675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:06.968665 15675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:06.968815 15675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:06.976876 15675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:45829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=41425
--tserver_master_addrs=127.1.194.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:06.978425 15675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:06.979758 15675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:06.992417 15713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:06.992913 15675 server_base.cc:1034] running on GCE node
W20260808 10:42:06.992430 15714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:06.995667 15716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:06.996326 15675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:06.997179 15675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:06.998333 15675 hybrid_clock.cc:648] HybridClock initialized: now 1786185726998268 us; error 84 us; skew 500 ppm
Aug 08 10:42:06 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:07.002135 15675 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:07.003559 15675 webserver.cc:460] Webserver started at http://127.1.194.65:41425/ using document root <none> and password file <none>
I20260808 10:42:07.004259 15675 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:07.004359 15675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:07.033105 15675 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.000s
I20260808 10:42:07.036029 15724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.037774 15675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:07.037979 15675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "52945ce182ce4fd0a998f9db91dfc9d0"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "0e95a2a0cc6b85b3278ec53be6aae0dd"
server_key_iv: "4fb9a7169442c51d22aaaf57a34fb503"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:07.038586 15675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:07.050451 15675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:07.053618 15675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:07.053831 15675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:07.054473 15675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:07.055454 15675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:07.055533 15675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.055624 15675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:07.055658 15675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.068186 15675 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:45829
I20260808 10:42:07.069552 15675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:07.070416 15844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:45829 every 8 connection(s)
I20260808 10:42:07.077636 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15675
I20260808 10:42:07.078127 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:44705
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:34321
--webserver_port=38529
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:07.090950 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.071856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:58453 (local address 127.1.194.126:34321)
0808 10:42:07.073914 (+ 2058us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:07.073935 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:07.073979 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:42:07.074274 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:07.074289 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:07.074708 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:07.075062 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:07.076703 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.078224 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.080030 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.080826 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.081684 (+ 858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.084169 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:07.084178 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:07.084198 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:07.084228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:07.086652 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.087368 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.087374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.087380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.087543 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.087965 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.087968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.087969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.088487 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:07.088645 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:07.089371 (+ 726us) server_negotiation.cc:299] Negotiation successful
0808 10:42:07.089969 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":116,"threads_started":1}
I20260808 10:42:07.090961 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.072006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34321 (local address 127.1.194.65:58453)
0808 10:42:07.072515 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:07.072565 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:07.073382 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:07.075482 (+ 2100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:07.075488 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:07.075856 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:07.076495 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:07.076516 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.078391 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.078397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:07.079858 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:07.079874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.081047 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.081051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:07.081254 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.082056 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:07.082078 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:07.083945 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:07.086806 (+ 2861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:07.086813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:07.086829 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:07.087191 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:07.087662 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:07.087666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:07.087668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:07.087843 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:07.088628 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:07.088636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:07.088992 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 10:42:07.089269 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 10:42:07.089723 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260808 10:42:07.095276 15845 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34321
I20260808 10:42:07.095628 15845 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.096323 15845 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
W20260808 10:42:07.098814 15845 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:34321 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:42:07.099052 15845 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.099311 15845 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:07.100250 15845 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.100484 15845 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
W20260808 10:42:07.101285 15845 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:42:07.226719 15850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:07.227108 15850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:07.227176 15850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:07.233382 15850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:07.233500 15850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:07.233600 15850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:07.240350 15850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:44705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=38529
--tserver_master_addrs=127.1.194.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:07.241521 15850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:07.242697 15850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:07.254299 15856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:07.254299 15857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:07.254792 15860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:07.255216 15850 server_base.cc:1034] running on GCE node
I20260808 10:42:07.255702 15850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:07.256529 15850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:07.257728 15850 hybrid_clock.cc:648] HybridClock initialized: now 1786185727257693 us; error 53 us; skew 500 ppm
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:07.260769 15850 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:07.261920 15850 webserver.cc:460] Webserver started at http://127.1.194.66:38529/ using document root <none> and password file <none>
I20260808 10:42:07.262665 15850 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:07.262778 15850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:07.283177 15850 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:07.286036 15868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.287442 15850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:07.287593 15850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "8c374452862547feb2ced49786cf479a"
format_stamp: "Formatted at 2026-08-08 10:42:05 on dist-test-slave-ss4n"
server_key: "8236dea9535a48f0de2f74f52c7cdbfa"
server_key_iv: "6447562822e163e29c87bd077a566d26"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:07.288213 15850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:07.305298 15850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:07.309091 15850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:07.309381 15850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:07.310233 15850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:07.311381 15850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:07.311503 15850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.311594 15850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:07.311678 15850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.323858 15850 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:44705
I20260808 10:42:07.325289 15850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:07.328802 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15850
I20260808 10:42:07.329344 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.194.84:38553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:45529
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:34321
--webserver_port=33331
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:07.326201 15989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:44705 every 8 connection(s)
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:07.345260 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.329778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:58971 (local address 127.1.194.126:34321)
0808 10:42:07.329974 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:07.329981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:07.330326 (+ 345us) server_negotiation.cc:407] Connection header received
0808 10:42:07.331760 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:07.331764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:07.331820 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:07.331874 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:07.333652 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.334518 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.336508 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.336876 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.336954 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.340136 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:07.340140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:07.340143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:07.340173 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:07.342001 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.343521 (+ 1520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.343525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.343527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.343633 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.344061 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.344063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.344064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.344248 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:07.344312 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:07.344936 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 10:42:07.345086 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:42:07.346181 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.328599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34321 (local address 127.1.194.66:58971)
0808 10:42:07.330149 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:07.330205 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:07.331258 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:07.332282 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:07.332291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:07.332653 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:07.333437 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:07.333456 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.334976 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.334979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:07.336366 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:07.336377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.337166 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.337173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:07.337380 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.338097 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:07.338117 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:07.339925 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:07.342141 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:07.342147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:07.342178 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:07.342526 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:07.343750 (+ 1224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:07.343753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:07.343755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:07.343933 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:07.344371 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:07.344376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:07.344684 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 10:42:07.344878 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:42:07.345205 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"thread_start_us":160,"threads_started":1}
I20260808 10:42:07.350986 15991 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34321
I20260808 10:42:07.351269 15991 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.352015 15991 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
W20260808 10:42:07.353817 15991 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:34321 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:42:07.353948 15991 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.354143 15991 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:07.355180 15991 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.355356 15991 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
W20260808 10:42:07.356242 15991 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:42:07.493641 15994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:07.493986 15994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:07.494074 15994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:07.498744 15994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:07.498813 15994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:07.498898 15994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:07.505097 15994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.194.84:38553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:45529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=33331
--tserver_master_addrs=127.1.194.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:07.506270 15994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:07.507321 15994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:07.517107 16003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:07.517134 16000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:07.517598 16001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:07.517659 15994 server_base.cc:1034] running on GCE node
I20260808 10:42:07.518088 15994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:07.518800 15994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:07.520057 15994 hybrid_clock.cc:648] HybridClock initialized: now 1786185727520023 us; error 47 us; skew 500 ppm
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:07.523922 15994 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:07.524821 15994 webserver.cc:460] Webserver started at http://127.1.194.67:33331/ using document root <none> and password file <none>
I20260808 10:42:07.525591 15994 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:07.525763 15994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:07.559877 15994 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 10:42:07.562857 16011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.564320 15994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:07.564472 15994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "3321f97862384162bb1e207fa6862f5c"
format_stamp: "Formatted at 2026-08-08 10:42:06 on dist-test-slave-ss4n"
server_key: "2d7d5c0037692397a3d2af9c44dc7235"
server_key_iv: "8fd5e02467dcbeba9f7ceba071d5c834"
server_key_version: "kuduclusterkey@0"
I20260808 10:42:07.565045 15994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:07.598865 15994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:07.602737 15994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:07.603044 15994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:07.603909 15994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:07.605050 15994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:07.605176 15994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.605294 15994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:07.605379 15994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:07.618147 15994 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:45529
I20260808 10:42:07.618173 16128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:45529 every 8 connection(s)
I20260808 10:42:07.619369 15994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:07.625563 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 15994
Aug 08 10:42:07 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:07.641649 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.623189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:39717 (local address 127.1.194.126:34321)
0808 10:42:07.623327 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:07.623332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:07.623383 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:42:07.626920 (+ 3537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:07.626924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:07.626983 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:07.627165 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:07.629021 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.629928 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.632053 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.632404 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.632499 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.636568 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:07.636572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:07.636575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:07.636604 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:07.638689 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.639460 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.639465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.639467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.639567 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.640034 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.640038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.640040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.640286 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:07.640365 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:07.641161 (+ 796us) server_negotiation.cc:299] Negotiation successful
0808 10:42:07.641333 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:42:07.644935 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.622079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34321 (local address 127.1.194.67:39717)
0808 10:42:07.622744 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:07.622811 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:07.626651 (+ 3840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:07.627634 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:07.627642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:07.628038 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:07.628830 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:07.628851 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.630464 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.630467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:07.631894 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:07.631905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.632736 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.632740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:07.632943 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.633872 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:07.633893 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:07.636356 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:07.638887 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:07.638894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:07.638925 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:07.639297 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:07.639701 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:07.639704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:07.639706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:07.639891 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:07.640460 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:07.640467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:07.640787 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 10:42:07.642378 (+ 1591us) client_negotiation.cc:240] Negotiation successful
0808 10:42:07.642771 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":163,"threads_started":1}
I20260808 10:42:07.646828 16129 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34321
I20260808 10:42:07.647090 16129 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.647619 16129 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
W20260808 10:42:07.649811 16129 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:34321 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:42:07.649945 16129 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.650177 16129 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:07.651101 16129 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:07.651301 16129 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
W20260808 10:42:07.652122 16129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:42:07.652482 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:07.630166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.1.194.126:34321)
0808 10:42:07.634740 (+ 4574us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:07.634750 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:07.634764 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:42:07.634798 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:07.634801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:07.634847 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:07.634932 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:07.636599 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.637525 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.643676 (+ 6151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:07.644009 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:07.644157 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:07.646319 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:07.646323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:07.646326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:07.646354 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:07.649773 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.650434 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.650438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.650440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.650551 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:07.651144 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:07.651149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:07.651152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:07.651396 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:07.651839 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:07.652105 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 10:42:07.652256 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4501,"thread_start_us":147,"threads_started":1}
Aug 08 10:42:08 dist-test-slave-ss4n krb5kdc[14580](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:08.101581 15845 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:08.101888 15845 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:08.102974 15635 ts_manager.cc:194] Registered new tserver with Master: 52945ce182ce4fd0a998f9db91dfc9d0 (127.1.194.65:45829)
I20260808 10:42:08.104264 15635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:58453
I20260808 10:42:08.356460 15991 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:08.356789 15991 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:08.357558 15635 ts_manager.cc:194] Registered new tserver with Master: 8c374452862547feb2ced49786cf479a (127.1.194.66:44705)
I20260808 10:42:08.358723 15635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:58971
Aug 08 10:42:08 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:08.652349 16129 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:08.652647 16129 heartbeater.cc:511] Master 127.1.194.126:34321 requested a full tablet report, sending...
I20260808 10:42:08.653369 15635 ts_manager.cc:194] Registered new tserver with Master: 3321f97862384162bb1e207fa6862f5c (127.1.194.67:45529)
I20260808 10:42:08.654232 15635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:39717
I20260808 10:42:08.663251 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:42:08 dist-test-slave-ss4n krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for HTTP/127.1.194.84@KRBTEST.COM
I20260808 10:42:08.686605 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.665005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.1.194.126:34321)
0808 10:42:08.665622 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.665626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.665872 (+ 246us) server_negotiation.cc:407] Connection header received
0808 10:42:08.666335 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.666338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.666393 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.666495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:08.667408 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.668274 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.669529 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.669877 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.669998 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.680607 (+ 10609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:08.680612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:08.680615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:08.680651 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:08.683680 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:08.684300 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:08.684304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:08.684306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:08.684404 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:08.684761 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:08.684764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:08.684765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:08.685033 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:08.685221 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.686165 (+ 944us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.686300 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":105,"threads_started":1}
I20260808 10:42:08.691210 15635 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:42:08.702708 15635 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:42:08.735746 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.721959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45529 (local address 127.0.0.1:38596)
0808 10:42:08.729652 (+ 7693us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:08.729668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:08.729776 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:08.730588 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:08.730594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:08.730619 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:08.730767 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.730774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.732276 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.732280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.733873 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.733885 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.735180 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.735183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.735302 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.735339 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:42:08.735448 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 10:42:08.735583 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7621,"thread_start_us":84,"threads_started":1}
I20260808 10:42:08.736240 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.722972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.1.194.67:45529)
0808 10:42:08.724127 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.724136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.729883 (+ 5747us) server_negotiation.cc:407] Connection header received
0808 10:42:08.729972 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.729977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.730215 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.730382 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:08.730907 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.732147 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.734016 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.734872 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.734974 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.735117 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.735988 (+ 871us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.736095 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":111,"threads_started":1}
I20260808 10:42:08.740373 16044 tablet_service.cc:1467] Processing CreateTablet for tablet 9031adee980d4b22b9fdc1f4ef014359 (DEFAULT_TABLE table=test-table [id=9ff722c275944e25a854d9c2806a09f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:08.742830 16044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9031adee980d4b22b9fdc1f4ef014359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:08.749158 16161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.721632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:44705 (local address 127.0.0.1:39192)
0808 10:42:08.722929 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:08.722993 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:08.723141 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:08.744219 (+ 21078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:08.744223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:08.744247 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:08.744363 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.744370 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.745989 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.745993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.747241 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.747252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.748650 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.748654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.748764 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.748791 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:42:08.748874 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:42:08.748965 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":142,"threads_started":1}
I20260808 10:42:08.749832 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.721728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.1.194.66:44705)
0808 10:42:08.743744 (+ 22016us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.743751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.743778 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:42:08.743857 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.743863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.744067 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.744241 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:08.744547 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.745832 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.747390 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.748298 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.748411 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.748575 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.749536 (+ 961us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.749652 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21890,"thread_start_us":136,"threads_started":1}
I20260808 10:42:08.753060 15908 tablet_service.cc:1467] Processing CreateTablet for tablet 9031adee980d4b22b9fdc1f4ef014359 (DEFAULT_TABLE table=test-table [id=9ff722c275944e25a854d9c2806a09f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:08.754346 15908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9031adee980d4b22b9fdc1f4ef014359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:08.763978 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.722502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:45829 (local address 127.0.0.1:56800)
0808 10:42:08.736700 (+ 14198us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:08.736713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:08.736791 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:08.749772 (+ 12981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:08.749776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:08.749795 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:08.749901 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.749910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.754893 (+ 4983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.754897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.756093 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.756107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.763486 (+ 7379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.763491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.763606 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.763639 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:42:08.763700 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:42:08.763790 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14138,"thread_start_us":82,"threads_started":1}
I20260808 10:42:08.765074 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.723131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.1.194.65:45829)
0808 10:42:08.725740 (+ 2609us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.725747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.737335 (+ 11588us) server_negotiation.cc:407] Connection header received
0808 10:42:08.737442 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.737448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.749594 (+ 12146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.750293 (+ 699us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:08.750468 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.751660 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.756361 (+ 4701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.758804 (+ 2443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.764560 (+ 5756us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.764736 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.764796 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.764899 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2492,"thread_start_us":105,"threads_started":1}
I20260808 10:42:08.765539 16166 tablet_bootstrap.cc:492] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Bootstrap starting.
I20260808 10:42:08.767985 16166 tablet_bootstrap.cc:654] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:08.768165 15775 tablet_service.cc:1467] Processing CreateTablet for tablet 9031adee980d4b22b9fdc1f4ef014359 (DEFAULT_TABLE table=test-table [id=9ff722c275944e25a854d9c2806a09f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:08.768931 16166 log.cc:824] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:08.769228 15775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9031adee980d4b22b9fdc1f4ef014359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:08.769913 16165 tablet_bootstrap.cc:492] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Bootstrap starting.
I20260808 10:42:08.772166 16166 tablet_bootstrap.cc:492] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: No bootstrap required, opened a new log
I20260808 10:42:08.772361 16166 ts_tablet_manager.cc:1397] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:42:08.775666 16165 tablet_bootstrap.cc:654] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:08.775668 16166 raft_consensus.cc:348] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.775976 16166 raft_consensus.cc:374] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:08.776032 16166 raft_consensus.cc:729] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c374452862547feb2ced49786cf479a, State: Initialized, Role: FOLLOWER
I20260808 10:42:08.776468 16166 consensus_queue.cc:260] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.776599 16165 log.cc:824] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:08.777470 16166 ts_tablet_manager.cc:1428] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:42:08.777881 15991 heartbeater.cc:503] Master 127.1.194.126:34321 was elected leader, sending a full tablet report...
I20260808 10:42:08.783720 16170 tablet_bootstrap.cc:492] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: Bootstrap starting.
I20260808 10:42:08.786373 16170 tablet_bootstrap.cc:654] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:08.786612 16165 tablet_bootstrap.cc:492] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: No bootstrap required, opened a new log
I20260808 10:42:08.786887 16165 ts_tablet_manager.cc:1397] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260808 10:42:08.787886 16170 log.cc:824] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:08.789958 16170 tablet_bootstrap.cc:492] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: No bootstrap required, opened a new log
I20260808 10:42:08.790201 16170 ts_tablet_manager.cc:1397] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:42:08.791512 16165 raft_consensus.cc:348] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.791858 16165 raft_consensus.cc:374] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:08.791962 16165 raft_consensus.cc:729] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3321f97862384162bb1e207fa6862f5c, State: Initialized, Role: FOLLOWER
I20260808 10:42:08.792479 16165 consensus_queue.cc:260] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.793630 16165 ts_tablet_manager.cc:1428] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:42:08.794123 16129 heartbeater.cc:503] Master 127.1.194.126:34321 was elected leader, sending a full tablet report...
I20260808 10:42:08.794832 16170 raft_consensus.cc:348] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.795404 16170 raft_consensus.cc:374] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:08.795522 16170 raft_consensus.cc:729] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52945ce182ce4fd0a998f9db91dfc9d0, State: Initialized, Role: FOLLOWER
I20260808 10:42:08.796061 16170 consensus_queue.cc:260] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.797220 16170 ts_tablet_manager.cc:1428] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260808 10:42:08.797631 15845 heartbeater.cc:503] Master 127.1.194.126:34321 was elected leader, sending a full tablet report...
W20260808 10:42:08.822549 15847 tablet.cc:2369] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:42:08.827492 15992 tablet.cc:2369] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:42:08.863755 16172 raft_consensus.cc:482] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:08.864008 16172 raft_consensus.cc:504] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.865464 16172 leader_election.cc:288] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c374452862547feb2ced49786cf479a (127.1.194.66:44705), 52945ce182ce4fd0a998f9db91dfc9d0 (127.1.194.65:45829)
W20260808 10:42:08.871706 16130 tablet.cc:2369] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:42:08.872481 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.866351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:40053 (local address 127.1.194.65:45829)
0808 10:42:08.866467 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.866472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.866488 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:42:08.866547 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.866550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.866597 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.866688 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:08.868781 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.869551 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.871179 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.871748 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.871851 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.871903 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.872257 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.872347 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:42:08.873217 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.865805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:45829 (local address 127.1.194.67:40053)
0808 10:42:08.866275 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:08.866292 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:08.866418 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:08.868400 (+ 1982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:08.868404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:08.868431 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:08.868591 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.868599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.869714 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.869718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.870984 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.870995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.871998 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.872001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.872108 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.872131 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:42:08.872904 (+ 773us) client_negotiation.cc:240] Negotiation successful
0808 10:42:08.873024 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":22,"thread_start_us":117,"threads_started":1}
I20260808 10:42:08.874053 15796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9031adee980d4b22b9fdc1f4ef014359" candidate_uuid: "3321f97862384162bb1e207fa6862f5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" is_pre_election: true
I20260808 10:42:08.874413 15796 raft_consensus.cc:2454] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3321f97862384162bb1e207fa6862f5c in term 0.
I20260808 10:42:08.875835 16013 leader_election.cc:302] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3321f97862384162bb1e207fa6862f5c, 52945ce182ce4fd0a998f9db91dfc9d0; no voters:
I20260808 10:42:08.876174 16172 raft_consensus.cc:2781] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:42:08.876266 16172 raft_consensus.cc:482] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:42:08.876307 16172 raft_consensus.cc:3037] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:08.878306 16172 raft_consensus.cc:504] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.878702 16172 leader_election.cc:288] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [CANDIDATE]: Term 1 election: Requested vote from peers 8c374452862547feb2ced49786cf479a (127.1.194.66:44705), 52945ce182ce4fd0a998f9db91dfc9d0 (127.1.194.65:45829)
I20260808 10:42:08.879299 15796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9031adee980d4b22b9fdc1f4ef014359" candidate_uuid: "3321f97862384162bb1e207fa6862f5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52945ce182ce4fd0a998f9db91dfc9d0"
I20260808 10:42:08.879439 15796 raft_consensus.cc:3037] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:08.880854 15796 raft_consensus.cc:2454] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3321f97862384162bb1e207fa6862f5c in term 1.
I20260808 10:42:08.881312 16013 leader_election.cc:302] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3321f97862384162bb1e207fa6862f5c, 52945ce182ce4fd0a998f9db91dfc9d0; no voters:
I20260808 10:42:08.881573 16172 raft_consensus.cc:2781] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:08.881812 16172 raft_consensus.cc:686] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 1 LEADER]: Becoming Leader. State: Replica: 3321f97862384162bb1e207fa6862f5c, State: Running, Role: LEADER
I20260808 10:42:08.882225 16172 consensus_queue.cc:237] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } }
I20260808 10:42:08.885911 15634 catalog_manager.cc:5495] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3321f97862384162bb1e207fa6862f5c (127.1.194.67). New cstate: current_term: 1 leader_uuid: "3321f97862384162bb1e207fa6862f5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3321f97862384162bb1e207fa6862f5c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 } health_report { overall_health: UNKNOWN } } }
I20260808 10:42:08.888855 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.866357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:39663 (local address 127.1.194.66:44705)
0808 10:42:08.866483 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.866488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.870600 (+ 4112us) server_negotiation.cc:407] Connection header received
0808 10:42:08.872615 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.872620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.872686 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.872742 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:08.873163 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.873899 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.885621 (+ 11722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.886326 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.886442 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.886499 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.888595 (+ 2096us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.888710 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:42:08.889288 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.865906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:44705 (local address 127.1.194.67:39663)
0808 10:42:08.870493 (+ 4587us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:08.870518 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:08.872445 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:08.872878 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:08.872881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:08.872900 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:08.873026 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.873032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.874034 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.874039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.875094 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:08.875109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.886616 (+ 11507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.886620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:08.886737 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.886762 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:42:08.889024 (+ 2262us) client_negotiation.cc:240] Negotiation successful
0808 10:42:08.889125 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4514,"mutex_wait_us":2,"thread_start_us":127,"threads_started":1}
I20260808 10:42:08.890110 15930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9031adee980d4b22b9fdc1f4ef014359" candidate_uuid: "3321f97862384162bb1e207fa6862f5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c374452862547feb2ced49786cf479a"
I20260808 10:42:08.890092 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9031adee980d4b22b9fdc1f4ef014359" candidate_uuid: "3321f97862384162bb1e207fa6862f5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c374452862547feb2ced49786cf479a" is_pre_election: true
I20260808 10:42:08.890447 15920 raft_consensus.cc:2454] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3321f97862384162bb1e207fa6862f5c in term 0.
I20260808 10:42:08.925915 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:08.920690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.1.194.67:45529)
0808 10:42:08.920812 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:08.920816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:08.920831 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:42:08.920918 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:08.920921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:08.920989 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:08.921128 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:08.921421 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.922260 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.923513 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:08.923885 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:08.923990 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:08.924440 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:08.925050 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:42:08.925191 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:08.925671 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 10:42:08.925738 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:42:08.932255 15920 raft_consensus.cc:3037] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:08.932446 15796 raft_consensus.cc:1261] T 9031adee980d4b22b9fdc1f4ef014359 P 52945ce182ce4fd0a998f9db91dfc9d0 [term 1 FOLLOWER]: Refusing update from remote peer 3321f97862384162bb1e207fa6862f5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:08.933168 16177 consensus_queue.cc:1035] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "52945ce182ce4fd0a998f9db91dfc9d0" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 45829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:08.933943 15920 raft_consensus.cc:1261] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 1 FOLLOWER]: Refusing update from remote peer 3321f97862384162bb1e207fa6862f5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:08.934808 16177 consensus_queue.cc:1035] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c374452862547feb2ced49786cf479a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 44705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:08.947057 16180 mvcc.cc:205] Tried to move back new op lower bound from 7316216745699352576 to 7316216745504739328. Current Snapshot: MvccSnapshot[applied={T|T < 7316216745699352576}]
I20260808 10:42:08.950167 16181 mvcc.cc:205] Tried to move back new op lower bound from 7316216745699352576 to 7316216745504739328. Current Snapshot: MvccSnapshot[applied={T|T < 7316216745699352576}]
I20260808 10:42:08.960142 15635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:08.960322 15635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:08.960578 16183 mvcc.cc:205] Tried to move back new op lower bound from 7316216745699352576 to 7316216745504739328. Current Snapshot: MvccSnapshot[applied={T|T < 7316216745699352576}]
I20260808 10:42:08.963105 15635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 590ad46526064e858637a890549e3566: Sending DeleteTablet for 3 replicas of tablet 9031adee980d4b22b9fdc1f4ef014359
I20260808 10:42:08.964512 15908 tablet_service.cc:1514] Processing DeleteTablet for tablet 9031adee980d4b22b9fdc1f4ef014359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:08 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39192
I20260808 10:42:08.965034 15775 tablet_service.cc:1514] Processing DeleteTablet for tablet 9031adee980d4b22b9fdc1f4ef014359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:08 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:56800
I20260808 10:42:08.966632 16044 tablet_service.cc:1514] Processing DeleteTablet for tablet 9031adee980d4b22b9fdc1f4ef014359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:08 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:38596
I20260808 10:42:08.967324 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15675
I20260808 10:42:08.967825 16189 tablet_replica.cc:330] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: stopping tablet replica
I20260808 10:42:08.968102 16189 raft_consensus.cc:2229] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:08.968361 16189 raft_consensus.cc:2258] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:08.973703 16191 tablet_replica.cc:330] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: stopping tablet replica
I20260808 10:42:08.974046 16191 raft_consensus.cc:2229] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:42:08.974421 16191 raft_consensus.cc:2258] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:08.980522 16189 ts_tablet_manager.cc:1905] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:08.980538 16191 ts_tablet_manager.cc:1905] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:08.983793 16191 ts_tablet_manager.cc:1918] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260808 10:42:08.984364 15620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:45829 (error 108)
I20260808 10:42:08.984405 16189 ts_tablet_manager.cc:1918] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:08.984503 16189 log.cc:1196] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/9031adee980d4b22b9fdc1f4ef014359
I20260808 10:42:08.984833 16189 ts_tablet_manager.cc:1939] T 9031adee980d4b22b9fdc1f4ef014359 P 8c374452862547feb2ced49786cf479a: Deleting consensus metadata
I20260808 10:42:08.985405 16191 log.cc:1196] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/9031adee980d4b22b9fdc1f4ef014359
I20260808 10:42:08.985720 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15850
I20260808 10:42:08.988994 16191 ts_tablet_manager.cc:1939] T 9031adee980d4b22b9fdc1f4ef014359 P 3321f97862384162bb1e207fa6862f5c: Deleting consensus metadata
I20260808 10:42:08.990628 15619 catalog_manager.cc:4841] TS 3321f97862384162bb1e207fa6862f5c (127.1.194.67:45529): tablet 9031adee980d4b22b9fdc1f4ef014359 (table test-table [id=9ff722c275944e25a854d9c2806a09f2]) successfully deleted
I20260808 10:42:09.000459 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:09.000026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:45829 (local address 127.0.0.1:56816)
0808 10:42:09.000267 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:09.000355 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:45829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260808 10:42:09.000839 15620 catalog_manager.cc:4584] TS 52945ce182ce4fd0a998f9db91dfc9d0 (127.1.194.65:45829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9031adee980d4b22b9fdc1f4ef014359: Network error: Client connection negotiation failed: client connection to 127.1.194.65:45829: connect: Connection refused (error 111)
I20260808 10:42:09.001268 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15994
I20260808 10:42:09.007449 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:09.007153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:44705 (local address 127.0.0.1:39208)
0808 10:42:09.007276 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:09.007343 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.66:44705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:42:09.011607 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 15601
I20260808 10:42:09.020145 1801 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:42:09.365850 1801 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:42:09.365984 1801 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:42:09.711185 1801 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:42:09.711 UTC [14613] LOG: received smart shutdown request
2026-08-08 10:42:09.716 UTC [14613] LOG: background worker "logical replication launcher" (PID 14623) exited with exit code 1
2026-08-08 10:42:09.716 UTC [14617] LOG: shutting down
2026-08-08 10:42:09.912 UTC [14613] LOG: database system is shut down
2026-08-08T10:42:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52030 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:42:10 dist-test-slave-ss4n krb5kdc[16201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:42:10 dist-test-slave-ss4n krb5kdc[16201](info): set up 2 sockets
Aug 08 10:42:10 dist-test-slave-ss4n krb5kdc[16201](info): commencing operation
krb5kdc: starting...
W20260808 10:42:11.135349 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:42:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:42:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:42:11.266717 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45567
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45567,127.1.194.125:43521,127.1.194.124:36369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:11.383730 16214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:11.384070 16214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:11.384155 16214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:11.388329 16214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:11.388423 16214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:11.388463 16214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:11.388536 16214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:11.388577 16214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:11.394155 16214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45567,127.1.194.125:43521,127.1.194.124:36369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:11.395263 16214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:11.396314 16214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:11.405205 16222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:11.405179 16219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:11.405298 16214 server_base.cc:1034] running on GCE node
W20260808 10:42:11.405198 16220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:11.406049 16214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:11.407145 16214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:11.408319 16214 hybrid_clock.cc:648] HybridClock initialized: now 1786185731408298 us; error 32 us; skew 500 ppm
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:11.411201 16214 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:11.412135 16214 webserver.cc:460] Webserver started at http://127.1.194.126:34897/ using document root <none> and password file <none>
I20260808 10:42:11.412773 16214 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:11.412839 16214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:11.413123 16214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:11.415256 16214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "a132d00765cd403186beca5e925936c6"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "65eb1f76be530cb38e3985e393674754"
server_key_iv: "9062edd7df49d1494a4edf9d4a829823"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.415849 16214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "a132d00765cd403186beca5e925936c6"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "65eb1f76be530cb38e3985e393674754"
server_key_iv: "9062edd7df49d1494a4edf9d4a829823"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.419643 16214 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:42:11.422026 16230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:11.423223 16214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:11.423368 16214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "a132d00765cd403186beca5e925936c6"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "65eb1f76be530cb38e3985e393674754"
server_key_iv: "9062edd7df49d1494a4edf9d4a829823"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.423480 16214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:11.466034 16214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:11.469103 16214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:11.469324 16214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:11.476516 16214 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:45567
I20260808 10:42:11.476528 16284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:45567 every 8 connection(s)
I20260808 10:42:11.477461 16214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:11.480173 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:11.483824 16285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.484265 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16214
I20260808 10:42:11.484397 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:42:11.484704 1801 external_mini_cluster.cc:1428] Setting key 4fc1355c94792699a413afc9b94d6d7e
WARNING: no policy specified for kudu/127.1.194.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.125@KRBTEST.COM" created.
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:11.502784 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.486442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45567 (local address 127.0.0.1:35984)
0808 10:42:11.487022 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.487074 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.487881 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.488533 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.488538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.488980 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.489293 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.489309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.490522 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.490525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.491255 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.491264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.491901 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.491905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.492012 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.492635 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.492650 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.497025 (+ 4375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.499549 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.499557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.499565 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.499801 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.500302 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.500305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.500307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.500419 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.501247 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.501253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.501381 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.501617 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.501902 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"spinlock_wait_cycles":23936,"thread_start_us":223,"threads_started":1}
I20260808 10:42:11.502789 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.486443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35984 (local address 127.1.194.126:45567)
0808 10:42:11.486973 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.486990 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.487183 (+ 193us) server_negotiation.cc:407] Connection header received
0808 10:42:11.488167 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.488177 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.488355 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.488536 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.489439 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.490395 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.491375 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.491742 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.491901 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.497211 (+ 5310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.497216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.497228 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.497251 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.499415 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.499921 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.499930 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.499935 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.500092 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.500529 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.500536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.500540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.501124 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.501270 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.501708 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.502061 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":406,"thread_start_us":139,"threads_started":1}
I20260808 10:42:11.505383 16285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.506095 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.505863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49378)
0808 10:42:11.505974 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.506040 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:42:11.506289 16231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
I20260808 10:42:11.507104 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.506907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49386)
0808 10:42:11.507001 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.507060 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:42:11.507288 16285 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:43521: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab.
Entry for principal kudu/127.1.194.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab.
I20260808 10:42:11.514724 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:43521
--webserver_interface=127.1.194.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab
--principal=kudu/127.1.194.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45567,127.1.194.125:43521,127.1.194.124:36369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:11.556560 16285 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } attempt: 1
I20260808 10:42:11.557543 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.557209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49402)
0808 10:42:11.557373 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.557438 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 10:42:11.558306 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.558046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49406)
0808 10:42:11.558177 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.558236 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:42:11.558473 16285 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:43521: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
I20260808 10:42:11.626653 16285 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } attempt: 2
I20260808 10:42:11.627521 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.627197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49408)
0808 10:42:11.627349 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.627412 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:42:11.628211 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.628012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49422)
0808 10:42:11.628112 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.628156 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:42:11.628352 16285 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:43521: Network error: Client connection negotiation failed: client connection to 127.1.194.125:43521: connect: Connection refused (error 111)
W20260808 10:42:11.631910 16292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:11.632158 16292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:11.632252 16292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:11.636293 16292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:11.636348 16292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:11.636366 16292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:11.636430 16292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:11.636453 16292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:11.642343 16292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45567,127.1.194.125:43521,127.1.194.124:36369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab
--principal=kudu/127.1.194.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.194.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:11.643370 16292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:11.644371 16292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:11.652972 16297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:11.653033 16298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:11.652959 16300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:11.653329 16292 server_base.cc:1034] running on GCE node
I20260808 10:42:11.653898 16292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:11.654973 16292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:11.656150 16292 hybrid_clock.cc:648] HybridClock initialized: now 1786185731656138 us; error 25 us; skew 500 ppm
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:11.658957 16292 init.cc:377] Logged in from keytab as kudu/127.1.194.125@KRBTEST.COM (short username kudu)
I20260808 10:42:11.659826 16292 webserver.cc:460] Webserver started at http://127.1.194.125:46149/ using document root <none> and password file <none>
I20260808 10:42:11.660374 16292 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:11.660465 16292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:11.660693 16292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:11.662743 16292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data/instance:
uuid: "f5711fadc53a4a38a81d61f3b8dcffa0"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "981619a32d71fa701f3b6ac79429f0ad"
server_key_iv: "0b5d2382f88704144265303cd6491d57"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.663314 16292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal/instance:
uuid: "f5711fadc53a4a38a81d61f3b8dcffa0"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "981619a32d71fa701f3b6ac79429f0ad"
server_key_iv: "0b5d2382f88704144265303cd6491d57"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.666906 16292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:11.669332 16308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:11.670579 16292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:11.670727 16292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal
uuid: "f5711fadc53a4a38a81d61f3b8dcffa0"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "981619a32d71fa701f3b6ac79429f0ad"
server_key_iv: "0b5d2382f88704144265303cd6491d57"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.670838 16292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:11.680047 16292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:11.683082 16292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:11.683288 16292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:11.690465 16292 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.125:43521
I20260808 10:42:11.690488 16362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.125:43521 every 8 connection(s)
I20260808 10:42:11.691421 16292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
I20260808 10:42:11.694186 16363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:11.698776 16363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.701746 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16292
I20260808 10:42:11.701895 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-1/wal/instance
I20260808 10:42:11.702206 1801 external_mini_cluster.cc:1428] Setting key b23c3389075bd05a351140edbe03da87
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.124@KRBTEST.COM; defaulting to no policy
I20260808 10:42:11.713675 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.700703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.1.194.126:45567)
0808 10:42:11.700884 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.700889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.701817 (+ 928us) server_negotiation.cc:407] Connection header received
0808 10:42:11.702883 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.702887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.702947 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.703065 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.704193 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.704740 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.705950 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.706211 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.706328 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.709218 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.709224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.709231 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.709257 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.710913 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.711660 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.711663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.711680 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.711788 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.712264 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.712268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.712270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.712585 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.712676 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.713277 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.713468 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:42:11.714555 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.701090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45567 (local address 127.0.0.1:35988)
0808 10:42:11.701666 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.701710 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.702667 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.703225 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.703232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.703581 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.704045 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.704057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.704877 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.704885 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.705826 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.705836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.706383 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.706387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.706559 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.707207 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.707224 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.709039 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.711107 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.711113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.711129 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.711475 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.711905 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.711909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.711910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.712128 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.712707 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.712713 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.712976 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.713147 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.713576 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.124@KRBTEST.COM" created.
I20260808 10:42:11.716095 16363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.719511 16285 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } attempt: 3
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
I20260808 10:42:11.727903 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.716730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49424)
0808 10:42:11.716908 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.716922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.716999 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.717607 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.717611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.717826 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.717975 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.717983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.718960 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.718963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.719681 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.719692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.720490 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.720494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.720588 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.721270 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.721282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.722679 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.726219 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.726225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.726229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.726436 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.726802 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.726804 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.726806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.726881 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.727433 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.727435 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.727512 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.727609 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.727746 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"spinlock_wait_cycles":115456}
I20260808 10:42:11.728161 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.716786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49424 (local address 127.1.194.125:43521)
0808 10:42:11.717223 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.717231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.717247 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:42:11.717277 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.717281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.717419 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.717591 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.718106 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.718835 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.719815 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.720368 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.720497 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.722835 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.722844 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.722856 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.722877 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.726074 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.726544 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.726549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.726550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.726690 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.727002 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.727004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.727005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.727328 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.727443 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.727748 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.727896 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":89,"threads_started":1}
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
I20260808 10:42:11.730351 16363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.731160 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.730856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46762)
0808 10:42:11.731002 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.731086 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 10:42:11.731381 16309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
I20260808 10:42:11.732363 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.732122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46778)
0808 10:42:11.732237 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.732305 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:42:11.732522 16363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:36369: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
I20260808 10:42:11.732851 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.719945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:49440)
0808 10:42:11.720166 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.720185 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.720272 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.720791 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.720794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.721008 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.721669 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.721682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.722964 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.722968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.723912 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.723924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.724911 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.724915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.725040 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.725617 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.725634 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.728889 (+ 3255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.731040 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.731043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.731045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.731296 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.731627 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.731630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.731632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.731749 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.732304 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.732307 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.732414 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.732599 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.732742 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 10:42:11.733274 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.720002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.1.194.125:43521)
0808 10:42:11.720531 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.720542 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.720559 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:42:11.720595 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.720600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.720642 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.721807 (+ 1165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.721918 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.722833 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.724038 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.724771 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.724906 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.729063 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.729067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.729068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.729087 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.730923 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.731397 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.731404 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.731406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.731495 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.731861 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.731864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.731866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.732194 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.732305 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.733068 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.733170 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":101,"threads_started":1}
I20260808 10:42:11.733913 16285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.734620 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.734331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46784)
0808 10:42:11.734450 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.734533 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 10:42:11.735363 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.735069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46800)
0808 10:42:11.735231 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.735289 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260808 10:42:11.735507 16285 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:36369: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab.
Entry for principal kudu/127.1.194.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab.
I20260808 10:42:11.737243 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:36369
--webserver_interface=127.1.194.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab
--principal=kudu/127.1.194.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:34315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45567,127.1.194.125:43521,127.1.194.124:36369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:11.766709 16285 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } attempt: 1
I20260808 10:42:11.767828 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.767386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46812)
0808 10:42:11.767645 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.767726 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260808 10:42:11.768697 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.768426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46822)
0808 10:42:11.768558 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.768624 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:42:11.768869 16285 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:36369: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
I20260808 10:42:11.781766 16363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } attempt: 1
I20260808 10:42:11.782658 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.782362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46828)
0808 10:42:11.782511 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.782581 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 10:42:11.783578 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.783370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46830)
0808 10:42:11.783479 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.783528 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:42:11.783818 16363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:36369: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
I20260808 10:42:11.844069 16285 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } attempt: 2
I20260808 10:42:11.844972 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.844658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46840)
0808 10:42:11.844822 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.844885 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 10:42:11.845904 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.845670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46842)
0808 10:42:11.845788 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.845842 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 10:42:11.846076 16285 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:36369: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
I20260808 10:42:11.852002 16363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } attempt: 2
I20260808 10:42:11.852826 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.852513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46846)
0808 10:42:11.852669 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.852751 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 10:42:11.853732 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.853391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:46848)
0808 10:42:11.853628 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.853683 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
W20260808 10:42:11.853847 16363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:36369: Network error: Client connection negotiation failed: client connection to 127.1.194.124:36369: connect: Connection refused (error 111)
W20260808 10:42:11.854614 16371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:11.854849 16371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:11.854890 16371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:11.858959 16371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:11.859023 16371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:11.859045 16371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:11.859062 16371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:11.859117 16371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:11.864794 16371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45567,127.1.194.125:43521,127.1.194.124:36369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab
--principal=kudu/127.1.194.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.194.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:11.865981 16371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:11.867034 16371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:11.875728 16376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:11.875756 16379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:11.875769 16377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:11.875890 16371 server_base.cc:1034] running on GCE node
I20260808 10:42:11.876543 16371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:11.877710 16371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:11.878907 16371 hybrid_clock.cc:648] HybridClock initialized: now 1786185731878890 us; error 29 us; skew 500 ppm
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:11.881855 16371 init.cc:377] Logged in from keytab as kudu/127.1.194.124@KRBTEST.COM (short username kudu)
I20260808 10:42:11.882737 16371 webserver.cc:460] Webserver started at http://127.1.194.124:33263/ using document root <none> and password file <none>
I20260808 10:42:11.883298 16371 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:11.883389 16371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:11.883622 16371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:11.885736 16371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data/instance:
uuid: "7913f698cd4143b28cc4b682ac4b6bc9"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "518af71086f9126f7aa1b1acddfad197"
server_key_iv: "686fa30255650bb275a46e5f2cbe2ace"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.886291 16371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal/instance:
uuid: "7913f698cd4143b28cc4b682ac4b6bc9"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "518af71086f9126f7aa1b1acddfad197"
server_key_iv: "686fa30255650bb275a46e5f2cbe2ace"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.889941 16371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:11.892522 16386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:11.893883 16371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:11.894049 16371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal
uuid: "7913f698cd4143b28cc4b682ac4b6bc9"
format_stamp: "Formatted at 2026-08-08 10:42:11 on dist-test-slave-ss4n"
server_key: "518af71086f9126f7aa1b1acddfad197"
server_key_iv: "686fa30255650bb275a46e5f2cbe2ace"
server_key_version: "encryptionkey@0"
I20260808 10:42:11.894176 16371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:11.910125 16371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:11.913689 16371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:11.913925 16371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:11.922374 16371 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.124:36369
I20260808 10:42:11.922389 16441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.124:36369 every 8 connection(s)
I20260808 10:42:11.923338 16371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
I20260808 10:42:11.924450 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16371
I20260808 10:42:11.924679 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185564023815-1801-0/minicluster-data/master-2/wal/instance
I20260808 10:42:11.924968 1801 external_mini_cluster.cc:1428] Setting key 7ba0dd3aacd33845508b9b86f7d0fbbd
I20260808 10:42:11.926512 16442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:11.930783 16442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:11.940124 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.926341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.1.194.124:36369)
0808 10:42:11.926726 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.926746 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.926804 (+ 58us) server_negotiation.cc:407] Connection header received
0808 10:42:11.927626 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.927644 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.927997 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.928280 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.929195 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.930299 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.931431 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.931942 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.932070 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.934954 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.934959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.934967 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.934993 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.936808 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.937256 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.937266 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.937271 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.937377 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.937774 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.937779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.937782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.938265 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.938465 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.938762 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.939081 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":137,"threads_started":1}
I20260808 10:42:11.942138 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.932542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45567 (local address 127.0.0.1:40114)
0808 10:42:11.932906 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.932943 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.933097 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.933393 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.933398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.933782 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.933966 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.933977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.934881 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.934884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.935684 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.935694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.936310 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.936313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.936398 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.936924 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.936936 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.938402 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.940283 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.940294 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.940301 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.940593 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.940906 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.940914 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.940915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.940987 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.941388 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.941394 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.941624 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.941774 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.941948 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":162,"threads_started":1}
I20260808 10:42:11.942595 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.932602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.1.194.126:45567)
0808 10:42:11.932746 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.932751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.933038 (+ 287us) server_negotiation.cc:407] Connection header received
0808 10:42:11.933201 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.933204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.933258 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.933347 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.934131 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.934746 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.935812 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.936185 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.936313 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.938543 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.938547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.938553 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.938583 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.940160 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.940704 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.940707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.940709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.940793 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.941081 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.941085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.941087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.941277 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.941390 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.941928 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.942433 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:42:11.943544 16442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.945046 16363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } attempt: 3
I20260808 10:42:11.945284 16285 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } attempt: 3
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:42:11.953245 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.942391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.1.194.126:45567)
0808 10:42:11.942702 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.942720 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.942735 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:42:11.942872 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.942875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.942919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.943008 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.943744 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.944441 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.945266 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.945641 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.945735 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.949042 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.949047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.949052 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.949077 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.950964 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.951466 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.951472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.951474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.951561 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.951877 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.951880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.951881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.952107 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.952302 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.952919 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.953039 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260808 10:42:11.954699 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.943982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:43521 (local address 127.0.0.1:36694)
0808 10:42:11.944095 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.944108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.944194 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.944479 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.944481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.944642 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.944724 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.944728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.946218 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.946221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.947037 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.947049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.947861 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.947868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.947973 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.948431 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.948449 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.949890 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.952768 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.952772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.952775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.953234 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.953597 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.953600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.953602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.953688 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.954204 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.954208 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.954299 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.954410 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.954538 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"spinlock_wait_cycles":1792}
I20260808 10:42:11.955320 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.944130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.1.194.125:43521)
0808 10:42:11.944283 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.944288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.944307 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:42:11.944342 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.944344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.944381 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.944466 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.944842 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.945573 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.947187 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.947611 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.947719 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.950014 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.950020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.950024 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.950054 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.952635 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.953356 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.953360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.953362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.953442 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.953795 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.953799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.953802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.954111 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.954204 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.954461 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.955067 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:42:11.955863 16442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:42:11.956249 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.945380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:55592)
0808 10:42:11.945739 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.945753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.945813 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.946341 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.946344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.946528 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.946616 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.946623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.948304 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.948308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.949136 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.949147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.949807 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.949811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.949896 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.950338 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.950357 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.951760 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.954134 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.954138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.954140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.954394 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.955139 (+ 745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.955142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.955145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.955266 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.955712 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.955718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.955824 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.955937 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.956085 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280}
I20260808 10:42:11.957273 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.945944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.1.194.124:36369)
0808 10:42:11.946097 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.946101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.946114 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:42:11.946146 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.946149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.946200 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.946320 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.947073 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.948182 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.949289 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.949692 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.950109 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.951902 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.951906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.951908 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.951934 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.954026 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.954944 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.954947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.954948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.955031 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.955388 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.955394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.955398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.955605 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.955708 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.957060 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.957166 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:42:11.957649 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.945651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:55600)
0808 10:42:11.945824 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.945836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.945898 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.946695 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.946698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.946863 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.946930 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.946934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.948078 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.948082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.949176 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.949188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.950805 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.950812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.951169 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.951657 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.951671 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.953032 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.956012 (+ 2980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.956015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.956017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.956249 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.956598 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.956604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.956609 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.956705 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.957129 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.957132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.957221 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.957331 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.957462 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 10:42:11.957711 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.946037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55600 (local address 127.1.194.124:36369)
0808 10:42:11.946507 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.946514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.946524 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:42:11.946554 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.946556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.946584 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.946679 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.947063 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.947956 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.949700 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.950668 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.951369 (+ 701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.953180 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.953184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.953186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.953223 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.955893 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.956382 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.956385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.956386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.956486 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.956832 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.956835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.956837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.957016 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.957112 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.957414 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.957548 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"spinlock_wait_cycles":7424,"thread_start_us":104,"threads_started":1}
I20260808 10:42:11.960947 16363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: Bootstrap starting.
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
Aug 08 10:42:11 dist-test-slave-ss4n krb5kdc[16201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:42:11.964107 16363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:11.964818 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.954601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.1.194.125:43521)
0808 10:42:11.954705 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.954711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.954726 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:42:11.954760 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.954764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.954809 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.954898 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.956617 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.957246 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.958295 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.958607 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.958695 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.961687 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.961691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.961694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.961720 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.963127 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.963627 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.963630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.963632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.963718 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.964003 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.964008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.964010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.964186 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.964382 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.964558 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.964672 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:42:11.965080 16363 log.cc:824] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:11.966464 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.956314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:36369 (local address 127.0.0.1:55606)
0808 10:42:11.956541 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:11.956557 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:11.956622 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:11.958209 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:11.958215 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:11.958362 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:11.958461 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.958471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.959316 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.959322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.959821 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:11.959829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.960234 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.960237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:11.960341 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.960746 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:11.960765 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:11.962216 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:11.964676 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.964678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.964680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.965152 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.965436 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:11.965442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:11.965453 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:11.965588 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:11.966015 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:11.966017 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:11.966092 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 10:42:11.966194 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:42:11.966314 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":23}
I20260808 10:42:11.967087 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.956513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.1.194.124:36369)
0808 10:42:11.957974 (+ 1461us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.957978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.957991 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:42:11.958019 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.958023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.958085 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.958557 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.958624 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.959205 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.959944 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.960135 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.960237 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.962374 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.962379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.962384 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.962413 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.964566 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.965258 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.965260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.965262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.965334 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.965705 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.965708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.965709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.965892 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.965981 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.966836 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.966950 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1407,"thread_start_us":118,"threads_started":1}
I20260808 10:42:11.967696 16363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: No bootstrap required, opened a new log
I20260808 10:42:11.968086 16285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6: Bootstrap starting.
I20260808 10:42:11.968222 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:42:11.970356 16285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:11.971313 16285 log.cc:824] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:11.971917 16363 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:11.972183 16363 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:11.972239 16363 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5711fadc53a4a38a81d61f3b8dcffa0, State: Initialized, Role: FOLLOWER
I20260808 10:42:11.972718 16363 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:11.972983 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: Bootstrap starting.
I20260808 10:42:11.974081 16363 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:11.975664 16442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:11.976500 16442 log.cc:824] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:11.978384 16460 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:11.978506 16460 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:42:11.978241 16292 master_runner.cc:186] Error getting master registration for 127.1.194.126:45567: OK, instance_id { permanent_uuid: "a132d00765cd403186beca5e925936c6" instance_seqno: 1786185731474436 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:42:11.981590 16292 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:42:11.982637 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.971575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.1.194.125:43521)
0808 10:42:11.971683 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.971687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.972312 (+ 625us) server_negotiation.cc:407] Connection header received
0808 10:42:11.972500 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.972503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.972551 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.972624 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.973784 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.974569 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.975991 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.976304 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.976367 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.978516 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.978519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.978522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.978546 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.980365 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.980853 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.980856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.980859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.980938 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.981247 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.981250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.981252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.981466 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.981684 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.982365 (+ 681us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.982479 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:42:11.982836 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.971329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.1.194.124:36369)
0808 10:42:11.971432 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.971436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.972681 (+ 1245us) server_negotiation.cc:407] Connection header received
0808 10:42:11.972843 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.972849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.972894 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.972940 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.973856 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.974571 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.975959 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.976275 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.976354 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.978520 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.978523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.978526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.978547 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.980722 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.981183 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.981189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.981191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.981271 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.981896 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.981899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.981901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.982121 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.982242 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.982595 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.982692 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 10:42:11.985146 16472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:42:11.985217 16472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:42:11.985774 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: No bootstrap required, opened a new log
I20260808 10:42:11.986780 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:11.971934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.1.194.126:45567)
0808 10:42:11.972054 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:11.972058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:11.976929 (+ 4871us) server_negotiation.cc:407] Connection header received
0808 10:42:11.977448 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:11.977451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:11.977557 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:11.977612 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:11.978843 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.979614 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.980728 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:11.981088 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:11.981162 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:11.982787 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:11.982791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:11.982794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:11.982818 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:11.984419 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.984875 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.984878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.984880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.984983 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:11.985468 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:11.985474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:11.985528 (+ 54us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:11.985752 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:11.985873 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:11.986504 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 10:42:11.986634 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:42:11.986969 16285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6: No bootstrap required, opened a new log
I20260808 10:42:11.989856 16442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:11.990120 16442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:11.990190 16442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7913f698cd4143b28cc4b682ac4b6bc9, State: Initialized, Role: FOLLOWER
I20260808 10:42:11.990119 16285 raft_consensus.cc:348] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:11.990342 16285 raft_consensus.cc:374] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:11.990410 16285 raft_consensus.cc:729] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a132d00765cd403186beca5e925936c6, State: Initialized, Role: FOLLOWER
I20260808 10:42:11.990671 16442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:11.990813 16285 consensus_queue.cc:260] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:11.992014 16442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:11.992054 16285 sys_catalog.cc:564] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:11.992112 16475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:11.992234 16475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:42:11.992471 16476 sys_catalog.cc:455] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:11.992617 16476 sys_catalog.cc:458] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:42:11.998934 16499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:42:11.999032 16499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 10:42:12.000118 16493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:42:12.000202 16493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:42:12.002585 16214 master_runner.cc:263] No leader master found from master a132d00765cd403186beca5e925936c6
I20260808 10:42:12.002653 16214 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:42:12.003997 16371 master_runner.cc:263] No leader master found from master 7913f698cd4143b28cc4b682ac4b6bc9
I20260808 10:42:12.004076 16371 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:42:12.017130 16476 raft_consensus.cc:482] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:12.017313 16476 raft_consensus.cc:504] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:12.018270 16476 leader_election.cc:288] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5711fadc53a4a38a81d61f3b8dcffa0 (127.1.194.125:43521), 7913f698cd4143b28cc4b682ac4b6bc9 (127.1.194.124:36369)
I20260808 10:42:12.019018 16337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a132d00765cd403186beca5e925936c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" is_pre_election: true
I20260808 10:42:12.019313 16337 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a132d00765cd403186beca5e925936c6 in term 0.
I20260808 10:42:12.019714 16415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a132d00765cd403186beca5e925936c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" is_pre_election: true
I20260808 10:42:12.019989 16415 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a132d00765cd403186beca5e925936c6 in term 0.
I20260808 10:42:12.020433 16231 leader_election.cc:302] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a132d00765cd403186beca5e925936c6, f5711fadc53a4a38a81d61f3b8dcffa0; no voters:
I20260808 10:42:12.020921 16476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:42:12.021008 16476 raft_consensus.cc:482] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:42:12.021061 16476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:12.022418 16476 raft_consensus.cc:504] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:12.022742 16476 leader_election.cc:288] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [CANDIDATE]: Term 1 election: Requested vote from peers f5711fadc53a4a38a81d61f3b8dcffa0 (127.1.194.125:43521), 7913f698cd4143b28cc4b682ac4b6bc9 (127.1.194.124:36369)
I20260808 10:42:12.023255 16415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a132d00765cd403186beca5e925936c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7913f698cd4143b28cc4b682ac4b6bc9"
I20260808 10:42:12.023375 16415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:12.024580 16415 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a132d00765cd403186beca5e925936c6 in term 1.
I20260808 10:42:12.024883 16337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a132d00765cd403186beca5e925936c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0"
I20260808 10:42:12.025002 16337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:12.026759 16337 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a132d00765cd403186beca5e925936c6 in term 1.
I20260808 10:42:12.027070 16231 leader_election.cc:302] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a132d00765cd403186beca5e925936c6, f5711fadc53a4a38a81d61f3b8dcffa0; no voters:
I20260808 10:42:12.029667 16476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:12.029912 16476 raft_consensus.cc:686] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [term 1 LEADER]: Becoming Leader. State: Replica: a132d00765cd403186beca5e925936c6, State: Running, Role: LEADER
I20260808 10:42:12.030268 16476 consensus_queue.cc:237] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } }
I20260808 10:42:12.032024 16476 sys_catalog.cc:455] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a132d00765cd403186beca5e925936c6. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.032132 16476 sys_catalog.cc:458] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:12.037029 16504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:12.040560 16504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:12.047057 16415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [term 1 FOLLOWER]: Refusing update from remote peer a132d00765cd403186beca5e925936c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:12.047760 16502 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:12.055531 16475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a132d00765cd403186beca5e925936c6. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.055658 16475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:42:12.056694 16337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [term 1 FOLLOWER]: Refusing update from remote peer a132d00765cd403186beca5e925936c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:12.057284 16502 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:12.061717 16460 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a132d00765cd403186beca5e925936c6. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.061852 16460 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:42:12.070452 16502 sys_catalog.cc:455] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.070609 16502 sys_catalog.cc:458] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:12.071692 16460 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.071810 16460 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:42:12.072165 16505 mvcc.cc:205] Tried to move back new op lower bound from 7316216758455951360 to 7316216758399102976. Current Snapshot: MvccSnapshot[applied={T|T < 7316216758455951360}]
I20260808 10:42:12.072916 16508 mvcc.cc:205] Tried to move back new op lower bound from 7316216758455951360 to 7316216758399102976. Current Snapshot: MvccSnapshot[applied={T|T < 7316216758455951360}]
I20260808 10:42:12.074990 16476 sys_catalog.cc:455] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.075124 16476 sys_catalog.cc:458] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:12.076212 16475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a132d00765cd403186beca5e925936c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a132d00765cd403186beca5e925936c6" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45567 } } peers { permanent_uuid: "f5711fadc53a4a38a81d61f3b8dcffa0" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 43521 } } peers { permanent_uuid: "7913f698cd4143b28cc4b682ac4b6bc9" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 36369 } } }
I20260808 10:42:12.076342 16475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:42:12.078944 16506 mvcc.cc:205] Tried to move back new op lower bound from 7316216758455951360 to 7316216758399102976. Current Snapshot: MvccSnapshot[applied={T|T < 7316216758455951360}]
I20260808 10:42:12.080931 16504 catalog_manager.cc:1347] Generated new cluster ID: f48e09bf62f24f6aa70916b6f29e5203
I20260808 10:42:12.080996 16504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:12.119995 16504 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:42:12.121289 16504 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:12.138293 16504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a132d00765cd403186beca5e925936c6: Generated new TSK 0
I20260808 10:42:12.139135 16504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:42:12.986613 16472 catalog_manager.cc:1259] Loaded cluster ID: f48e09bf62f24f6aa70916b6f29e5203
I20260808 10:42:12.986748 16472 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: loading cluster ID for follower catalog manager: success
I20260808 10:42:12.988435 16472 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: acquiring CA information for follower catalog manager: success
I20260808 10:42:12.989634 16472 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5711fadc53a4a38a81d61f3b8dcffa0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:42:13.000218 16499 catalog_manager.cc:1259] Loaded cluster ID: f48e09bf62f24f6aa70916b6f29e5203
I20260808 10:42:13.000315 16499 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: loading cluster ID for follower catalog manager: success
I20260808 10:42:13.002084 16499 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: acquiring CA information for follower catalog manager: success
I20260808 10:42:13.003351 16499 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7913f698cd4143b28cc4b682ac4b6bc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:42:13.211450 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16214
I20260808 10:42:13.222036 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16292
I20260808 10:42:13.228716 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16371
2026-08-08T10:42:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3201 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:42:13 dist-test-slave-ss4n krb5kdc[16521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:42:13 dist-test-slave-ss4n krb5kdc[16521](info): set up 2 sockets
Aug 08 10:42:13 dist-test-slave-ss4n krb5kdc[16521](info): commencing operation
krb5kdc: starting...
W20260808 10:42:14.384217 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:42:14 dist-test-slave-ss4n krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:42:14.521399 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:41965
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:41965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:14.632977 16534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:14.633239 16534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:14.633281 16534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:14.637462 16534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:14.637545 16534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:14.637565 16534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:14.637581 16534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:14.637598 16534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:14.642954 16534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:41965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:41965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:14.644047 16534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:14.645097 16534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:14.653719 16540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:14.653733 16539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:14.653827 16534 server_base.cc:1034] running on GCE node
W20260808 10:42:14.653719 16543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:14.654577 16534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:14.655699 16534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:14.656896 16534 hybrid_clock.cc:648] HybridClock initialized: now 1786185734656864 us; error 42 us; skew 500 ppm
Aug 08 10:42:14 dist-test-slave-ss4n krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185734, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:14.665361 16534 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:14.666429 16534 webserver.cc:460] Webserver started at http://127.1.194.126:36871/ using document root <none> and password file <none>
I20260808 10:42:14.667028 16534 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:14.667125 16534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:14.667366 16534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:14.669420 16534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "d29b26530e074f0d979bbbff9a6dd420"
format_stamp: "Formatted at 2026-08-08 10:42:14 on dist-test-slave-ss4n"
server_key: "f74c35aac5c4c44478eb5f07ba8cd94f"
server_key_iv: "dc44a45e63fc15c2f738566b6835f220"
server_key_version: "encryptionkey@0"
I20260808 10:42:14.670049 16534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "d29b26530e074f0d979bbbff9a6dd420"
format_stamp: "Formatted at 2026-08-08 10:42:14 on dist-test-slave-ss4n"
server_key: "f74c35aac5c4c44478eb5f07ba8cd94f"
server_key_iv: "dc44a45e63fc15c2f738566b6835f220"
server_key_version: "encryptionkey@0"
I20260808 10:42:14.674227 16534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:42:14.677028 16551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:14.678249 16534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 10:42:14.678400 16534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "d29b26530e074f0d979bbbff9a6dd420"
format_stamp: "Formatted at 2026-08-08 10:42:14 on dist-test-slave-ss4n"
server_key: "f74c35aac5c4c44478eb5f07ba8cd94f"
server_key_iv: "dc44a45e63fc15c2f738566b6835f220"
server_key_version: "encryptionkey@0"
I20260808 10:42:14.678521 16534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:14.689930 16534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:14.692909 16534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:14.693121 16534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:14.700774 16534 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:41965
I20260808 10:42:14.700806 16604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:41965 every 8 connection(s)
I20260808 10:42:14.701763 16534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:14.704560 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:14.708635 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16534
I20260808 10:42:14.708787 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:42:14.709082 1801 external_mini_cluster.cc:1428] Setting key dd661f80efeeee6e52c1752d90a6f365
I20260808 10:42:14.710928 16605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420: Bootstrap starting.
I20260808 10:42:14.713219 16605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:14.714061 16605 log.cc:824] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:14.716392 16605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420: No bootstrap required, opened a new log
Aug 08 10:42:14 dist-test-slave-ss4n krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:14.720454 16605 raft_consensus.cc:348] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b26530e074f0d979bbbff9a6dd420" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41965 } }
I20260808 10:42:14.720789 16605 raft_consensus.cc:374] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:14.720891 16605 raft_consensus.cc:729] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d29b26530e074f0d979bbbff9a6dd420, State: Initialized, Role: FOLLOWER
I20260808 10:42:14.721390 16605 consensus_queue.cc:260] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b26530e074f0d979bbbff9a6dd420" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41965 } }
I20260808 10:42:14.721601 16605 raft_consensus.cc:388] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:14.721674 16605 raft_consensus.cc:482] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:14.721778 16605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:14.722777 16605 raft_consensus.cc:504] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b26530e074f0d979bbbff9a6dd420" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41965 } }
I20260808 10:42:14.723141 16605 leader_election.cc:302] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d29b26530e074f0d979bbbff9a6dd420; no voters:
I20260808 10:42:14.723510 16605 leader_election.cc:288] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:42:14.723587 16611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:14.723815 16611 raft_consensus.cc:686] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [term 1 LEADER]: Becoming Leader. State: Replica: d29b26530e074f0d979bbbff9a6dd420, State: Running, Role: LEADER
I20260808 10:42:14.724198 16611 consensus_queue.cc:237] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b26530e074f0d979bbbff9a6dd420" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41965 } }
I20260808 10:42:14.724858 16605 sys_catalog.cc:564] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:14.725019 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:14.710621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.1.194.126:41965)
0808 10:42:14.711176 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:14.711194 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:14.711235 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:42:14.711984 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:14.712001 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:14.712334 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:14.712706 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:14.713762 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:14.715093 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:14.716262 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:14.716824 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:14.717008 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:14.719575 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:14.719580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:14.719594 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:14.719626 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:14.721701 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:14.722216 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:14.722222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:14.722228 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:14.722355 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:14.722675 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:14.722678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:14.722680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:14.723163 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:14.723383 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:14.723806 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 10:42:14.724142 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":136,"threads_started":1}
I20260808 10:42:14.725421 16611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d29b26530e074f0d979bbbff9a6dd420. Latest consensus state: current_term: 1 leader_uuid: "d29b26530e074f0d979bbbff9a6dd420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b26530e074f0d979bbbff9a6dd420" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41965 } } }
I20260808 10:42:14.725564 16611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:14.725888 16612 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d29b26530e074f0d979bbbff9a6dd420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b26530e074f0d979bbbff9a6dd420" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41965 } } }
I20260808 10:42:14.725978 16612 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:14.726243 16616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:14.728734 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:42:14.730122 16616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:14.735550 16616 catalog_manager.cc:1347] Generated new cluster ID: baf12485819348a3ac381202cfb886e9
I20260808 10:42:14.735630 16616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:14.760696 16616 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:42:14.761719 16616 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:14.771845 16616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d29b26530e074f0d979bbbff9a6dd420: Generated new TSK 0
I20260808 10:42:14.772543 16616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:42:14.785449 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16534
2026-08-08T10:42:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1556 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:42:14 dist-test-slave-ss4n krb5kdc[16634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:42:14 dist-test-slave-ss4n krb5kdc[16634](info): set up 2 sockets
Aug 08 10:42:14 dist-test-slave-ss4n krb5kdc[16634](info): commencing operation
krb5kdc: starting...
W20260808 10:42:15.871559 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:42:15 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:42:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:42:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:42:16.002404 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34035
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:16.112808 16648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:16.113067 16648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:16.113118 16648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:16.117129 16648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:16.117189 16648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:16.117208 16648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:16.117225 16648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:16.117240 16648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:16.122506 16648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:16.123577 16648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:16.124647 16648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:16.133468 16654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.133486 16656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:16.133723 16648 server_base.cc:1034] running on GCE node
W20260808 10:42:16.133893 16653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:16.134443 16648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:16.135521 16648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:16.136720 16648 hybrid_clock.cc:648] HybridClock initialized: now 1786185736136686 us; error 40 us; skew 500 ppm
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:16.139657 16648 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:16.140605 16648 webserver.cc:460] Webserver started at http://127.1.194.126:43975/ using document root <none> and password file <none>
I20260808 10:42:16.141165 16648 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:16.141227 16648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:16.141469 16648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:16.143596 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "345cf8c2698d4d6dbc77dc26b444d6f4"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "7eff2c7c0e1ce3bbf1d58bff019954ce"
server_key_iv: "b2ccb0d076613c60be19461796ed8252"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.144166 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "345cf8c2698d4d6dbc77dc26b444d6f4"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "7eff2c7c0e1ce3bbf1d58bff019954ce"
server_key_iv: "b2ccb0d076613c60be19461796ed8252"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.147814 16648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 10:42:16.150301 16663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.151562 16648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 10:42:16.151739 16648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "345cf8c2698d4d6dbc77dc26b444d6f4"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "7eff2c7c0e1ce3bbf1d58bff019954ce"
server_key_iv: "b2ccb0d076613c60be19461796ed8252"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.151856 16648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:16.177722 16648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:16.180821 16648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:16.181031 16648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:16.188640 16648 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34035
I20260808 10:42:16.188937 16716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34035 every 8 connection(s)
I20260808 10:42:16.189679 16648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:16.192339 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:16.197697 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap starting.
I20260808 10:42:16.199506 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16648
I20260808 10:42:16.199651 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:42:16.199962 1801 external_mini_cluster.cc:1428] Setting key 54d506562436c991dbffa1d52bb37ee4
I20260808 10:42:16.200456 16717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:16.201310 16717 log.cc:824] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:16.203218 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: No bootstrap required, opened a new log
I20260808 10:42:16.206624 16717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:16.206889 16717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:16.206957 16717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345cf8c2698d4d6dbc77dc26b444d6f4, State: Initialized, Role: FOLLOWER
I20260808 10:42:16.207454 16717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:16.207582 16717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:16.207652 16717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:16.207769 16717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:16.208945 16717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:16.209323 16717 leader_election.cc:302] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345cf8c2698d4d6dbc77dc26b444d6f4; no voters:
I20260808 10:42:16.209755 16717 leader_election.cc:288] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:42:16.209867 16722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:16.210139 16722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 LEADER]: Becoming Leader. State: Replica: 345cf8c2698d4d6dbc77dc26b444d6f4, State: Running, Role: LEADER
I20260808 10:42:16.210474 16722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:16.211076 16717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:16.211752 16722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345cf8c2698d4d6dbc77dc26b444d6f4. Latest consensus state: current_term: 1 leader_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } } }
I20260808 10:42:16.211932 16722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:16.212283 16723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } } }
I20260808 10:42:16.212378 16723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:16.215340 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.201129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55600 (local address 127.1.194.126:34035)
0808 10:42:16.201679 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:16.201694 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:16.201727 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:42:16.202530 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:16.202550 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:16.202862 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:16.203190 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:16.203979 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.204937 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.205856 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.206322 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.206488 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.209516 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:16.209526 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:16.209550 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:16.209579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:16.211680 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.212225 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.212234 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.212241 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.212400 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.212796 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.212802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.212804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.213402 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:16.213659 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:16.214241 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 10:42:16.214547 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":163,"threads_started":1}
W20260808 10:42:16.216246 16738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:42:16.216372 16738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:42:16.216451 16732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:16.217290 16732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:16.223598 16732 catalog_manager.cc:1347] Generated new cluster ID: 4433cb84d78449f18f8ec584d7042ea8
I20260808 10:42:16.223713 16732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:16.233462 16732 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:42:16.234624 16732 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:16.242208 16732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Generated new TSK 0
I20260808 10:42:16.242903 16732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:42:16.281155 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34035
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:16.412222 16744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:16.412472 16744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:16.412515 16744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:16.416611 16744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:16.416675 16744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:16.416765 16744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:16.422099 16744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:16.423138 16744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:16.424218 16744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:16.433787 16750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.433772 16749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.433941 16752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:16.434113 16744 server_base.cc:1034] running on GCE node
I20260808 10:42:16.434537 16744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:16.435200 16744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:16.436386 16744 hybrid_clock.cc:648] HybridClock initialized: now 1786185736436374 us; error 33 us; skew 500 ppm
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:16.439141 16744 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:16.440027 16744 webserver.cc:460] Webserver started at http://127.1.194.65:34947/ using document root <none> and password file <none>
I20260808 10:42:16.440582 16744 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:16.440685 16744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:16.440945 16744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:16.443060 16744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "0a33c83603c5482f883ba856b670357d"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "eaff58c194d32bb861878776e62feccf"
server_key_iv: "7c5c3a5bd86855598cb95d896da6fb3d"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.443639 16744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "0a33c83603c5482f883ba856b670357d"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "eaff58c194d32bb861878776e62feccf"
server_key_iv: "7c5c3a5bd86855598cb95d896da6fb3d"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.447260 16744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:42:16.449610 16760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.450788 16744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:16.450924 16744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "0a33c83603c5482f883ba856b670357d"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "eaff58c194d32bb861878776e62feccf"
server_key_iv: "7c5c3a5bd86855598cb95d896da6fb3d"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.451038 16744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:16.466432 16744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:16.469313 16744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:16.469573 16744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:16.470175 16744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:16.470994 16744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:16.471068 16744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.471138 16744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:16.471181 16744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.480541 16744 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:38539
I20260808 10:42:16.480567 16881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:38539 every 8 connection(s)
I20260808 10:42:16.481463 16744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:16.489418 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16744
I20260808 10:42:16.489603 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:42:16.490003 1801 external_mini_cluster.cc:1428] Setting key c0d572ebbef901924badad5ccc05c6e5
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:16.495720 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.483501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:36005 (local address 127.1.194.126:34035)
0808 10:42:16.483686 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:16.483690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:16.484697 (+ 1007us) server_negotiation.cc:407] Connection header received
0808 10:42:16.485633 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:16.485635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:16.485684 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:16.485761 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:16.486954 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.487510 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.488611 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.488836 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.488914 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.492003 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:16.492007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:16.492009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:16.492032 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:16.493603 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.494155 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.494158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.494160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.494245 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.494596 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.494602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.494607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.494801 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:16.494911 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:16.495422 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 10:42:16.495554 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 10:42:16.496580 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.483863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:36005)
0808 10:42:16.484514 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:16.484555 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:16.485346 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:16.485932 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:16.485938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:16.486300 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:16.486820 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:16.486832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.487639 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.487642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:16.488503 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:16.488511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.488974 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.488977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:16.489174 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.489960 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:16.489982 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:16.491811 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:16.493737 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:16.493743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:16.493761 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:16.494037 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:16.494341 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:16.494344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:16.494346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:16.494498 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:16.494912 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:16.494920 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:16.495182 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 10:42:16.495382 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 10:42:16.495737 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":103,"threads_started":1}
I20260808 10:42:16.498143 16882 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34035
I20260808 10:42:16.498391 16882 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:16.498883 16882 heartbeater.cc:511] Master 127.1.194.126:34035 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:42:16.500619 16678 ts_manager.cc:194] Registered new tserver with Master: 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539)
I20260808 10:42:16.501969 16678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:36005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:42:16.521077 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34035
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:16.639067 16887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:16.639364 16887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:16.639477 16887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:16.644107 16887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:16.644183 16887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:16.644316 16887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:16.650503 16887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:16.651686 16887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:16.652839 16887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:16.663781 16895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.663780 16893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.663851 16892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:16.663887 16887 server_base.cc:1034] running on GCE node
I20260808 10:42:16.664450 16887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:16.665136 16887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:16.666311 16887 hybrid_clock.cc:648] HybridClock initialized: now 1786185736666294 us; error 32 us; skew 500 ppm
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:16.669157 16887 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:16.670121 16887 webserver.cc:460] Webserver started at http://127.1.194.66:40427/ using document root <none> and password file <none>
I20260808 10:42:16.670676 16887 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:16.670735 16887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:16.670972 16887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:16.672976 16887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "c6e504ba3739481c8685a3c393d36d2e"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "c57807c7956057346917e830bee3c3dd"
server_key_iv: "cbcb788ba3df8973de31bd5eaa723104"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.673621 16887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "c6e504ba3739481c8685a3c393d36d2e"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "c57807c7956057346917e830bee3c3dd"
server_key_iv: "cbcb788ba3df8973de31bd5eaa723104"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.677238 16887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:16.679674 16904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.680815 16887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:16.680949 16887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "c6e504ba3739481c8685a3c393d36d2e"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "c57807c7956057346917e830bee3c3dd"
server_key_iv: "cbcb788ba3df8973de31bd5eaa723104"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.681061 16887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:16.700984 16887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:16.704212 16887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:16.704376 16887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:16.704881 16887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:16.705719 16887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:16.705770 16887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.705809 16887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:16.705825 16887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.715277 16887 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:35853
I20260808 10:42:16.715338 17027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:35853 every 8 connection(s)
I20260808 10:42:16.716121 16887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:16.718760 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 16887
I20260808 10:42:16.718895 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:42:16.719257 1801 external_mini_cluster.cc:1428] Setting key ef522dedbf4a7d1e433dc21a94c9e9f7
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:42:16.730938 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.717925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:60573 (local address 127.1.194.126:34035)
0808 10:42:16.718054 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:16.718058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:16.719211 (+ 1153us) server_negotiation.cc:407] Connection header received
0808 10:42:16.720320 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:16.720323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:16.720377 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:16.720478 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:16.722104 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.722630 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.723851 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.724244 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.724368 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.727071 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:16.727077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:16.727084 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:16.727121 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:16.728853 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.729381 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.729387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.729390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.729514 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.729858 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.729864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.729868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.730105 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:16.730222 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:16.730709 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 10:42:16.730815 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:42:16.731772 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.718239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:60573)
0808 10:42:16.719034 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:16.719094 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:16.720112 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:16.720681 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:16.720691 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:16.721201 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:16.721935 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:16.721957 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.722765 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.722771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:16.723726 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:16.723736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.724390 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.724393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:16.724533 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.725260 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:16.725276 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:16.726885 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:16.728974 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:16.728979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:16.728991 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:16.729266 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:16.729622 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:16.729624 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:16.729626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:16.729760 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:16.730229 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:16.730237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:16.730480 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 10:42:16.730641 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 10:42:16.730899 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:42:16.733034 17028 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34035
I20260808 10:42:16.733268 17028 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:16.733784 17028 heartbeater.cc:511] Master 127.1.194.126:34035 requested a full tablet report, sending...
I20260808 10:42:16.734905 16678 ts_manager.cc:194] Registered new tserver with Master: c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
I20260808 10:42:16.735755 16678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:60573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:42:16.749137 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34035
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:16.861402 17033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:16.861681 17033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:16.861726 17033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:16.865756 17033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:16.865810 17033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:16.865882 17033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:16.871009 17033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:16.872002 17033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:16.872979 17033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:42:16.882853 17033 server_base.cc:1034] running on GCE node
W20260808 10:42:16.882858 17040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.882894 17039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:16.883047 17042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:16.883492 17033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:16.884145 17033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:16.885346 17033 hybrid_clock.cc:648] HybridClock initialized: now 1786185736885312 us; error 43 us; skew 500 ppm
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:16.888127 17033 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:16.889214 17033 webserver.cc:460] Webserver started at http://127.1.194.67:41171/ using document root <none> and password file <none>
I20260808 10:42:16.889838 17033 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:16.889930 17033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:16.890159 17033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:16.892169 17033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "a10bf3b095bc45c7864ab598ae9398e9"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "d55ddebf2a483d97ec5fb15fbb823ed7"
server_key_iv: "32f7f060f5317e2115bb807e326522b0"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.892774 17033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "a10bf3b095bc45c7864ab598ae9398e9"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "d55ddebf2a483d97ec5fb15fbb823ed7"
server_key_iv: "32f7f060f5317e2115bb807e326522b0"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.896569 17033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 10:42:16.899008 17050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.900413 17033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:42:16.900553 17033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "a10bf3b095bc45c7864ab598ae9398e9"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "d55ddebf2a483d97ec5fb15fbb823ed7"
server_key_iv: "32f7f060f5317e2115bb807e326522b0"
server_key_version: "encryptionkey@0"
I20260808 10:42:16.900678 17033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:16.933579 17033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:16.938515 17033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:16.938827 17033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:16.939682 17033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:16.940887 17033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:16.940972 17033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.941105 17033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:16.941150 17033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:16.952327 17033 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:45111
I20260808 10:42:16.952344 17169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:45111 every 8 connection(s)
I20260808 10:42:16.953240 17033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:16.956574 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17033
I20260808 10:42:16.956688 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:42:16.956935 1801 external_mini_cluster.cc:1428] Setting key ff77f495006217bdc6759b7591a814fd
Aug 08 10:42:16 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:16.967700 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.955134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:45077 (local address 127.1.194.126:34035)
0808 10:42:16.955294 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:16.955298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:16.957405 (+ 2107us) server_negotiation.cc:407] Connection header received
0808 10:42:16.958574 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:16.958577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:16.958649 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:16.958721 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:16.959974 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.960498 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.961642 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.961850 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.961929 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.964323 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:16.964326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:16.964328 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:16.964351 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:16.965695 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.966185 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.966189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.966191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.966273 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.966633 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.966637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.966638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.966816 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:16.966926 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:16.967414 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 10:42:16.967534 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:42:16.968465 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.955528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:45077)
0808 10:42:16.956298 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:16.957282 (+ 984us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:16.958365 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:16.958924 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:16.958933 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:16.959384 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:16.959850 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:16.959863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.960610 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.960612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:16.961472 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:16.961541 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.961977 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.961979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:16.962129 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.962689 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:16.962705 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:16.964152 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:16.965796 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:16.965801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:16.965816 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:16.966075 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:16.966366 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:16.966370 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:16.966372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:16.966513 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:16.966923 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:16.966930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:16.967174 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 10:42:16.967360 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 10:42:16.967689 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":140,"threads_started":1}
I20260808 10:42:16.969630 17170 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34035
I20260808 10:42:16.969892 17170 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:16.970399 17170 heartbeater.cc:511] Master 127.1.194.126:34035 requested a full tablet report, sending...
I20260808 10:42:16.971400 16678 ts_manager.cc:194] Registered new tserver with Master: a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111)
I20260808 10:42:16.971910 16678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:45077
I20260808 10:42:16.983031 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:42:16.992818 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:16.985140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55604 (local address 127.1.194.126:34035)
0808 10:42:16.985292 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:16.985297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:16.985522 (+ 225us) server_negotiation.cc:407] Connection header received
0808 10:42:16.985745 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:16.985749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:16.985800 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:16.985887 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:16.986607 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.987138 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.987973 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:16.988198 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:16.988303 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:16.989793 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:16.989797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:16.989799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:16.989830 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:16.991047 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.991491 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.991494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.991496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.991603 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:16.991890 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:16.991893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:16.991895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:16.992137 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:16.992278 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:16.992526 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 10:42:16.992646 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:42:16.995863 16678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:42:16.998862 16678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:42:17.013296 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.007144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.1.194.67:45111)
0808 10:42:17.008190 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.008206 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.008231 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:42:17.008312 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.008317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.008520 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.008688 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:17.009028 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.010280 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.011933 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.012587 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.012711 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.012864 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.013024 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.013120 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":149,"threads_started":1}
I20260808 10:42:17.013342 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.007070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45111 (local address 127.0.0.1:44044)
0808 10:42:17.007420 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.007465 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.007596 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.008687 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.008693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.008718 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:17.008890 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.008898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.010394 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.010397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.011784 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.011794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.012711 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.012715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.012821 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.012873 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.012988 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.013121 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260808 10:42:17.013355 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.007141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35853 (local address 127.0.0.1:44802)
0808 10:42:17.007832 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.007846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.007936 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.008907 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.008911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.008928 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:17.009031 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.009038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.010457 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.010461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.011706 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.011719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.012913 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.012917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.013002 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.013024 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.013077 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.013157 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":39,"thread_start_us":100,"threads_started":1}
I20260808 10:42:17.013697 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.008129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.1.194.65:38539)
0808 10:42:17.008892 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.008898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.008918 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:42:17.008985 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.008990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.009170 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.009319 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:17.009528 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.010230 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.012000 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.012775 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.012891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.013022 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.013408 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.013545 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":92,"threads_started":1}
I20260808 10:42:17.013861 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.008007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.1.194.66:35853)
0808 10:42:17.008482 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.008493 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.008516 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:42:17.008589 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.008593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.008771 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.008931 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:17.009161 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.010280 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.011852 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.012782 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.013423 (+ 641us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.013575 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.013623 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.013707 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":170,"threads_started":1}
I20260808 10:42:17.014565 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.007808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38539 (local address 127.0.0.1:50064)
0808 10:42:17.008517 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.008531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.008655 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.009300 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.009303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.009320 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:17.009398 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.009404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.010348 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.010355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.011889 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.011900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.013083 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.013086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.013160 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.013178 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.013245 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.013305 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":187,"thread_start_us":104,"threads_started":1}
I20260808 10:42:17.016304 17099 tablet_service.cc:1467] Processing CreateTablet for tablet d0964c331f7641baabcd0e4588d6ea28 (DEFAULT_TABLE table=test-table [id=74a5105057334d189c2d562a56ff1c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:17.016301 16958 tablet_service.cc:1467] Processing CreateTablet for tablet d0964c331f7641baabcd0e4588d6ea28 (DEFAULT_TABLE table=test-table [id=74a5105057334d189c2d562a56ff1c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:17.017189 16958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0964c331f7641baabcd0e4588d6ea28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:17.017196 17099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0964c331f7641baabcd0e4588d6ea28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:17.019946 16811 tablet_service.cc:1467] Processing CreateTablet for tablet d0964c331f7641baabcd0e4588d6ea28 (DEFAULT_TABLE table=test-table [id=74a5105057334d189c2d562a56ff1c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:17.020926 16811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0964c331f7641baabcd0e4588d6ea28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:17.023372 17188 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Bootstrap starting.
I20260808 10:42:17.024740 17189 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Bootstrap starting.
I20260808 10:42:17.025911 17188 tablet_bootstrap.cc:654] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:17.026623 17188 log.cc:824] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:17.026921 17189 tablet_bootstrap.cc:654] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:17.027741 17191 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Bootstrap starting.
I20260808 10:42:17.028486 17189 log.cc:824] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:17.028864 17188 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: No bootstrap required, opened a new log
I20260808 10:42:17.029105 17188 ts_tablet_manager.cc:1397] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 10:42:17.029867 17191 tablet_bootstrap.cc:654] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:17.030259 17189 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: No bootstrap required, opened a new log
I20260808 10:42:17.030453 17189 ts_tablet_manager.cc:1397] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:42:17.030654 17191 log.cc:824] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:17.032582 17191 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: No bootstrap required, opened a new log
I20260808 10:42:17.032804 17191 ts_tablet_manager.cc:1397] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:42:17.032959 17188 raft_consensus.cc:348] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.033255 17188 raft_consensus.cc:374] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:17.033341 17188 raft_consensus.cc:729] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6e504ba3739481c8685a3c393d36d2e, State: Initialized, Role: FOLLOWER
I20260808 10:42:17.033690 17189 raft_consensus.cc:348] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.033824 17188 consensus_queue.cc:260] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.033923 17189 raft_consensus.cc:374] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:17.033972 17189 raft_consensus.cc:729] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a10bf3b095bc45c7864ab598ae9398e9, State: Initialized, Role: FOLLOWER
I20260808 10:42:17.034370 17189 consensus_queue.cc:260] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.034606 17028 heartbeater.cc:503] Master 127.1.194.126:34035 was elected leader, sending a full tablet report...
I20260808 10:42:17.035327 17189 ts_tablet_manager.cc:1428] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:42:17.035663 17170 heartbeater.cc:503] Master 127.1.194.126:34035 was elected leader, sending a full tablet report...
I20260808 10:42:17.036144 17188 ts_tablet_manager.cc:1428] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:42:17.038239 17191 raft_consensus.cc:348] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.038501 17191 raft_consensus.cc:374] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:17.038565 17191 raft_consensus.cc:729] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a33c83603c5482f883ba856b670357d, State: Initialized, Role: FOLLOWER
I20260808 10:42:17.039019 17191 consensus_queue.cc:260] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.039698 16882 heartbeater.cc:503] Master 127.1.194.126:34035 was elected leader, sending a full tablet report...
I20260808 10:42:17.039955 17191 ts_tablet_manager.cc:1428] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260808 10:42:17.204253 17171 tablet.cc:2369] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:42:17.217252 17029 tablet.cc:2369] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:42:17.224247 17196 raft_consensus.cc:482] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:17.224443 17196 raft_consensus.cc:504] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.225880 17196 leader_election.cc:288] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853), 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539)
I20260808 10:42:17.229691 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.225850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35853 (local address 127.1.194.67:56531)
0808 10:42:17.226051 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.226070 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.226207 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.226532 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.226534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.226580 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:17.226711 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.226718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.227637 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.227641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.228471 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.228480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.229185 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.229189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.229309 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.229336 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.229419 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.229532 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260808 10:42:17.229691 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.226115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:56531 (local address 127.1.194.66:35853)
0808 10:42:17.226246 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.226251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.226274 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:42:17.226342 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.226345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.226400 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.226515 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:17.226880 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.227522 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.228607 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.229064 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.229161 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.229201 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.229444 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.229551 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:42:17.229837 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.226131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:45655 (local address 127.1.194.65:38539)
0808 10:42:17.226254 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.226259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.226612 (+ 353us) server_negotiation.cc:407] Connection header received
0808 10:42:17.226687 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.226691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.226742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.226830 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:17.227143 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.227670 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.228741 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.229197 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.229316 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.229361 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.229632 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.229716 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:42:17.229883 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.225850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38539 (local address 127.1.194.67:45655)
0808 10:42:17.226473 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.226490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.226600 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.226879 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.226882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.226901 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:17.227016 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.227022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.227783 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.227785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.228612 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.228622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.229327 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.229330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.229469 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.229521 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.229597 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.229661 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":215,"thread_start_us":134,"threads_started":1}
I20260808 10:42:17.230562 16832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0964c331f7641baabcd0e4588d6ea28" candidate_uuid: "a10bf3b095bc45c7864ab598ae9398e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a33c83603c5482f883ba856b670357d" is_pre_election: true
I20260808 10:42:17.230561 16978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0964c331f7641baabcd0e4588d6ea28" candidate_uuid: "a10bf3b095bc45c7864ab598ae9398e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6e504ba3739481c8685a3c393d36d2e" is_pre_election: true
I20260808 10:42:17.230883 16832 raft_consensus.cc:2454] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a10bf3b095bc45c7864ab598ae9398e9 in term 0.
I20260808 10:42:17.230883 16978 raft_consensus.cc:2454] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a10bf3b095bc45c7864ab598ae9398e9 in term 0.
I20260808 10:42:17.231382 17052 leader_election.cc:302] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a33c83603c5482f883ba856b670357d, a10bf3b095bc45c7864ab598ae9398e9; no voters:
I20260808 10:42:17.231676 17196 raft_consensus.cc:2781] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:42:17.231760 17196 raft_consensus.cc:482] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:42:17.231804 17196 raft_consensus.cc:3037] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Advancing to term 1
W20260808 10:42:17.232842 16883 tablet.cc:2369] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:42:17.232823 17196 raft_consensus.cc:504] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.233191 17196 leader_election.cc:288] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [CANDIDATE]: Term 1 election: Requested vote from peers c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853), 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539)
I20260808 10:42:17.233587 16978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0964c331f7641baabcd0e4588d6ea28" candidate_uuid: "a10bf3b095bc45c7864ab598ae9398e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6e504ba3739481c8685a3c393d36d2e"
I20260808 10:42:17.233604 16832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0964c331f7641baabcd0e4588d6ea28" candidate_uuid: "a10bf3b095bc45c7864ab598ae9398e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a33c83603c5482f883ba856b670357d"
I20260808 10:42:17.233718 16978 raft_consensus.cc:3037] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:17.233724 16832 raft_consensus.cc:3037] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:17.234706 16978 raft_consensus.cc:2454] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a10bf3b095bc45c7864ab598ae9398e9 in term 1.
I20260808 10:42:17.234791 16832 raft_consensus.cc:2454] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a10bf3b095bc45c7864ab598ae9398e9 in term 1.
I20260808 10:42:17.234992 17053 leader_election.cc:302] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e; no voters:
I20260808 10:42:17.235159 17196 raft_consensus.cc:2781] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:17.235392 17196 raft_consensus.cc:686] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 1 LEADER]: Becoming Leader. State: Replica: a10bf3b095bc45c7864ab598ae9398e9, State: Running, Role: LEADER
I20260808 10:42:17.235704 17196 consensus_queue.cc:237] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.238960 16678 catalog_manager.cc:5495] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "a10bf3b095bc45c7864ab598ae9398e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260808 10:42:17.305781 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.301867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.1.194.67:45111)
0808 10:42:17.302061 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.302065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.302085 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:42:17.302192 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.302194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.302255 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.302359 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:17.302608 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.303169 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.304041 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.304297 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.304383 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.304663 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:17.305163 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:42:17.305261 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.305417 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.305608 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 10:42:17.312050 16978 raft_consensus.cc:1261] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Refusing update from remote peer a10bf3b095bc45c7864ab598ae9398e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:17.312050 16832 raft_consensus.cc:1261] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Refusing update from remote peer a10bf3b095bc45c7864ab598ae9398e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:17.312846 17199 consensus_queue.cc:1035] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:17.313048 17196 consensus_queue.cc:1035] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:17.322101 17205 mvcc.cc:205] Tried to move back new op lower bound from 7316216780021940224 to 7316216779719909376. Current Snapshot: MvccSnapshot[applied={T|T < 7316216780021940224}]
I20260808 10:42:17.328938 16678 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:17.329137 16678 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:17.331617 16678 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Sending DeleteTablet for 3 replicas of tablet d0964c331f7641baabcd0e4588d6ea28
I20260808 10:42:17.332461 16811 tablet_service.cc:1514] Processing DeleteTablet for tablet d0964c331f7641baabcd0e4588d6ea28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:17 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:50064
I20260808 10:42:17.332460 16958 tablet_service.cc:1514] Processing DeleteTablet for tablet d0964c331f7641baabcd0e4588d6ea28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:17 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:44802
I20260808 10:42:17.332588 17099 tablet_service.cc:1514] Processing DeleteTablet for tablet d0964c331f7641baabcd0e4588d6ea28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:17 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:44044
I20260808 10:42:17.332679 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16744
I20260808 10:42:17.334406 17214 tablet_replica.cc:330] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: stopping tablet replica
I20260808 10:42:17.334813 17214 raft_consensus.cc:2229] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:42:17.335278 17213 tablet_replica.cc:330] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: stopping tablet replica
I20260808 10:42:17.335305 17214 raft_consensus.cc:2258] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:17.335659 17213 raft_consensus.cc:2229] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:17.336032 17213 raft_consensus.cc:2258] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:17.337670 17214 ts_tablet_manager.cc:1905] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:17.337672 17213 ts_tablet_manager.cc:1905] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:17.340456 17214 ts_tablet_manager.cc:1918] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:17.340631 17214 log.cc:1196] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/d0964c331f7641baabcd0e4588d6ea28
I20260808 10:42:17.341048 17214 ts_tablet_manager.cc:1939] T d0964c331f7641baabcd0e4588d6ea28 P a10bf3b095bc45c7864ab598ae9398e9: Deleting consensus metadata
I20260808 10:42:17.341117 17213 ts_tablet_manager.cc:1918] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:17.341300 17213 log.cc:1196] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/d0964c331f7641baabcd0e4588d6ea28
I20260808 10:42:17.341694 17213 ts_tablet_manager.cc:1939] T d0964c331f7641baabcd0e4588d6ea28 P c6e504ba3739481c8685a3c393d36d2e: Deleting consensus metadata
I20260808 10:42:17.342386 16665 catalog_manager.cc:4841] TS a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): tablet d0964c331f7641baabcd0e4588d6ea28 (table test-table [id=74a5105057334d189c2d562a56ff1c33]) successfully deleted
W20260808 10:42:17.342913 16665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:38539 (error 108)
I20260808 10:42:17.343080 16666 catalog_manager.cc:4841] TS c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): tablet d0964c331f7641baabcd0e4588d6ea28 (table test-table [id=74a5105057334d189c2d562a56ff1c33]) successfully deleted
I20260808 10:42:17.343849 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16887
I20260808 10:42:17.343973 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.343687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38539 (local address 127.0.0.1:50068)
0808 10:42:17.343799 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.343870 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:42:17.345467 16665 catalog_manager.cc:4584] TS 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0964c331f7641baabcd0e4588d6ea28: Network error: Client connection negotiation failed: client connection to 127.1.194.65:38539: connect: Connection refused (error 111)
I20260808 10:42:17.350523 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 17033
I20260808 10:42:17.356987 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 16648
I20260808 10:42:17.364073 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34035
--webserver_interface=127.1.194.126
--webserver_port=43975
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:17.475958 17216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:17.476298 17216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:17.476382 17216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:17.480460 17216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:17.480572 17216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:17.480608 17216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:17.480647 17216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:17.480702 17216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:17.486231 17216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=43975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:17.487375 17216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:17.488423 17216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:17.502326 17222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:17.502338 17226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:17.502331 17224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:17.502385 17216 server_base.cc:1034] running on GCE node
I20260808 10:42:17.503185 17216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:17.504438 17216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:17.505596 17216 hybrid_clock.cc:648] HybridClock initialized: now 1786185737505561 us; error 47 us; skew 500 ppm
Aug 08 10:42:17 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185737, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:17.508538 17216 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:17.509568 17216 webserver.cc:460] Webserver started at http://127.1.194.126:43975/ using document root <none> and password file <none>
I20260808 10:42:17.510180 17216 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:17.510286 17216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:17.514251 17216 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:17.516626 17233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:17.517953 17216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:17.518129 17216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "345cf8c2698d4d6dbc77dc26b444d6f4"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "7eff2c7c0e1ce3bbf1d58bff019954ce"
server_key_iv: "b2ccb0d076613c60be19461796ed8252"
server_key_version: "encryptionkey@0"
I20260808 10:42:17.518857 17216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:17.545789 17216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:17.549768 17216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:17.550071 17216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:17.558087 17216 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34035
I20260808 10:42:17.558390 17288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34035 every 8 connection(s)
I20260808 10:42:17.559352 17216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:17.561564 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17216
I20260808 10:42:17.562093 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:38539
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:34035
--webserver_port=34947
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:17.564843 17289 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:42:17.566761 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap starting.
I20260808 10:42:17.584374 17289 log.cc:824] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:17.593950 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:17.594385 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap complete.
I20260808 10:42:17.598677 17289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:17.599099 17289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345cf8c2698d4d6dbc77dc26b444d6f4, State: Initialized, Role: FOLLOWER
I20260808 10:42:17.599601 17289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:17.599725 17289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:17.599831 17289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:17.599928 17289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:42:17.601939 17289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:17.602317 17289 leader_election.cc:302] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345cf8c2698d4d6dbc77dc26b444d6f4; no voters:
I20260808 10:42:17.602682 17289 leader_election.cc:288] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:42:17.602777 17293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:42:17.602991 17293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 LEADER]: Becoming Leader. State: Replica: 345cf8c2698d4d6dbc77dc26b444d6f4, State: Running, Role: LEADER
I20260808 10:42:17.603360 17293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:17.604040 17289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:17.604494 17293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345cf8c2698d4d6dbc77dc26b444d6f4. Latest consensus state: current_term: 2 leader_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } } }
I20260808 10:42:17.604699 17293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:17.605465 17294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } } }
I20260808 10:42:17.605626 17294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:17.606541 17304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:17.611287 17304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:17.611899 17304 catalog_manager.cc:1259] Loaded cluster ID: 4433cb84d78449f18f8ec584d7042ea8
I20260808 10:42:17.611971 17304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:17.613394 17304 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 10:42:17.618320 17304 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 10:42:17.623821 17304 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:17.624574 17304 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Loaded TSK: 0
I20260808 10:42:17.625767 17304 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:42:17.678927 17291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:17.679225 17291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:17.679311 17291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:17.683494 17291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:17.683579 17291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:17.683666 17291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:17.689069 17291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:38539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=34947
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:17.690196 17291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:17.691296 17291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:17.702265 17318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:17.702332 17319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:17.702276 17321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:17.702600 17291 server_base.cc:1034] running on GCE node
I20260808 10:42:17.703164 17291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:17.703827 17291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:17.705001 17291 hybrid_clock.cc:648] HybridClock initialized: now 1786185737704963 us; error 53 us; skew 500 ppm
Aug 08 10:42:17 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185737, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:17.708180 17291 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:17.709192 17291 webserver.cc:460] Webserver started at http://127.1.194.65:34947/ using document root <none> and password file <none>
I20260808 10:42:17.709856 17291 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:17.709937 17291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:17.714078 17291 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 10:42:17.716430 17329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:17.717844 17291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:17.717979 17291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "0a33c83603c5482f883ba856b670357d"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "eaff58c194d32bb861878776e62feccf"
server_key_iv: "7c5c3a5bd86855598cb95d896da6fb3d"
server_key_version: "encryptionkey@0"
I20260808 10:42:17.718535 17291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:17.733031 17291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:17.736299 17291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:17.736511 17291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:17.737190 17291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:17.738351 17337 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:42:17.743250 17291 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:42:17.743330 17291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 10:42:17.743409 17291 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:42:17.745015 17291 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:42:17.745093 17291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:42:17.745378 17337 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Bootstrap starting.
I20260808 10:42:17.755550 17291 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:38539
I20260808 10:42:17.755792 17447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:38539 every 8 connection(s)
I20260808 10:42:17.756533 17291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:17.759891 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17291
I20260808 10:42:17.760389 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:35853
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:34035
--webserver_port=40427
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:17.767032 17337 log.cc:824] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:42:17 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185737, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:17.775640 17337 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:17.776113 17337 tablet_bootstrap.cc:492] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Bootstrap complete.
I20260808 10:42:17.777367 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.758547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:60051 (local address 127.1.194.126:34035)
0808 10:42:17.759030 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.759052 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.760746 (+ 1694us) server_negotiation.cc:407] Connection header received
0808 10:42:17.761016 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.761032 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.761408 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.761774 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:17.763716 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.764838 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.766664 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.767476 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.767702 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.771170 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:17.771180 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:17.771200 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:17.771241 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:17.773680 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:17.774325 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:17.774333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:17.774338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:17.774504 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:17.774903 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:17.774909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:17.774913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:17.775440 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:17.775593 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.776232 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.776526 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":113,"threads_started":1}
I20260808 10:42:17.777406 17451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.758580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:60051)
0808 10:42:17.759303 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.759354 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.760436 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.762293 (+ 1857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.762305 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.762835 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:17.763493 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.763515 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.765007 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.765014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.766498 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.766513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.767975 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.767981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.768234 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.768938 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:17.768965 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:17.770909 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:17.773822 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:17.773828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:17.773839 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:17.774186 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:17.774622 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:17.774626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:17.774628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:17.774791 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:17.775578 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:17.775584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:17.775860 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.776081 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.776411 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":143,"threads_started":1}
I20260808 10:42:17.777941 17337 ts_tablet_manager.cc:1397] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Time spent bootstrapping tablet: real 0.033s user 0.023s sys 0.008s
I20260808 10:42:17.779721 17448 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34035
I20260808 10:42:17.779950 17448 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:17.780400 17448 heartbeater.cc:511] Master 127.1.194.126:34035 requested a full tablet report, sending...
I20260808 10:42:17.780365 17337 raft_consensus.cc:348] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.780833 17337 raft_consensus.cc:729] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a33c83603c5482f883ba856b670357d, State: Initialized, Role: FOLLOWER
I20260808 10:42:17.781272 17337 consensus_queue.cc:260] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:17.782217 17337 ts_tablet_manager.cc:1428] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 10:42:17.783951 17250 ts_manager.cc:194] Registered new tserver with Master: 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539)
I20260808 10:42:17.787791 17250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:60051
I20260808 10:42:17.790181 17448 heartbeater.cc:503] Master 127.1.194.126:34035 was elected leader, sending a full tablet report...
Aug 08 10:42:17 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185737, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:42:17.797848 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.787139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38539 (local address 127.0.0.1:50080)
0808 10:42:17.787680 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.787723 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.787851 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.788444 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.788450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.788987 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:17.789114 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.789122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.790577 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.790582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.791225 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.791235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.791939 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.791942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.792053 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.792600 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:17.792615 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:17.794248 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:17.796173 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:17.796184 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:17.796189 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:17.796420 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:17.796724 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:17.796728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:17.796730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:17.796817 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:17.797265 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:17.797271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:17.797394 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.797564 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.797724 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":140,"threads_started":1}
I20260808 10:42:17.798362 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.787190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.1.194.65:38539)
0808 10:42:17.788078 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.788086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.788103 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:42:17.788153 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.788156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.788295 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.788449 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:17.789240 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.790455 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.791356 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.791816 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.791946 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.794393 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:17.794402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:17.794413 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:17.794433 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:17.796057 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:17.796545 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:17.796555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:17.796557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:17.796630 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:17.796915 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:17.796918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:17.796920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:17.797150 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:17.797289 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.798095 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.798251 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":108,"threads_started":1}
I20260808 10:42:17.799192 17379 tablet_service.cc:1514] Processing DeleteTablet for tablet d0964c331f7641baabcd0e4588d6ea28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:17 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:50080
I20260808 10:42:17.800132 17458 tablet_replica.cc:330] stopping tablet replica
I20260808 10:42:17.800411 17458 raft_consensus.cc:2229] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:17.800644 17458 raft_consensus.cc:2258] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:17.801666 17458 ts_tablet_manager.cc:1905] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:17.805286 17458 ts_tablet_manager.cc:1918] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:17.805397 17458 log.cc:1196] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/d0964c331f7641baabcd0e4588d6ea28
I20260808 10:42:17.805701 17458 ts_tablet_manager.cc:1939] T d0964c331f7641baabcd0e4588d6ea28 P 0a33c83603c5482f883ba856b670357d: Deleting consensus metadata
I20260808 10:42:17.807456 17235 catalog_manager.cc:4841] TS 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539): tablet d0964c331f7641baabcd0e4588d6ea28 (table test-table [id=74a5105057334d189c2d562a56ff1c33]) successfully deleted
W20260808 10:42:17.884346 17452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:17.884611 17452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:17.884697 17452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:17.888659 17452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:17.888738 17452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:17.888797 17452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:17.893873 17452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:35853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=40427
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:17.894913 17452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:17.895920 17452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:17.905966 17463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:17.905938 17464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:17.905974 17466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:17.906373 17452 server_base.cc:1034] running on GCE node
I20260808 10:42:17.906804 17452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:17.907503 17452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:17.908707 17452 hybrid_clock.cc:648] HybridClock initialized: now 1786185737908678 us; error 40 us; skew 500 ppm
Aug 08 10:42:17 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185737, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:17.911638 17452 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:17.912509 17452 webserver.cc:460] Webserver started at http://127.1.194.66:40427/ using document root <none> and password file <none>
I20260808 10:42:17.913089 17452 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:17.913208 17452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:17.920353 17452 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260808 10:42:17.923199 17474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:17.924762 17452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:42:17.924890 17452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "c6e504ba3739481c8685a3c393d36d2e"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "c57807c7956057346917e830bee3c3dd"
server_key_iv: "cbcb788ba3df8973de31bd5eaa723104"
server_key_version: "encryptionkey@0"
I20260808 10:42:17.925464 17452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:17.949997 17452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:17.953392 17452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:17.953634 17452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:17.954252 17452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:17.955088 17452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:17.955163 17452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:17.955236 17452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:17.955281 17452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:17.964761 17452 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:35853
I20260808 10:42:17.964780 17597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:35853 every 8 connection(s)
I20260808 10:42:17.965871 17452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:17.970585 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17452
I20260808 10:42:17.971141 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:45111
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:34035
--webserver_port=41171
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:42:17 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185737, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:17.980484 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.967730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:53105 (local address 127.1.194.126:34035)
0808 10:42:17.967885 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:17.967889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:17.968938 (+ 1049us) server_negotiation.cc:407] Connection header received
0808 10:42:17.969864 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:17.969868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:17.969929 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:17.970007 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:17.971638 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.972187 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.973344 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.973671 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.973787 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.976336 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:17.976342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:17.976348 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:17.976387 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:17.978247 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:17.978751 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:17.978757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:17.978759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:17.978862 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:17.979199 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:17.979205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:17.979209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:17.979482 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:17.979602 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:17.980057 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 10:42:17.980235 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:42:17.981092 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:17.968122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:53105)
0808 10:42:17.968779 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:17.968820 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:17.969653 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:17.970176 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:17.970182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:17.970772 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:17.971487 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.971504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.972324 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.972327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.973203 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:17.973211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:17.973827 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:17.973832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:17.974009 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:17.974652 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:17.974668 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:17.976139 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:17.978370 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:17.978374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:17.978386 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:17.978644 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:17.978960 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:17.978962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:17.978963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:17.979101 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:17.979588 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:17.979593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:17.979815 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 10:42:17.979966 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 10:42:17.980228 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":104,"threads_started":1}
I20260808 10:42:17.982299 17598 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34035
I20260808 10:42:17.982553 17598 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:17.983062 17598 heartbeater.cc:511] Master 127.1.194.126:34035 requested a full tablet report, sending...
I20260808 10:42:17.984138 17250 ts_manager.cc:194] Registered new tserver with Master: c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
I20260808 10:42:17.984707 17250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:53105
W20260808 10:42:18.085456 17601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:18.085870 17601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:18.085953 17601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:18.090083 17601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:18.090162 17601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:18.090248 17601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:18.095470 17601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:45111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=41171
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:18.096520 17601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:18.097604 17601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:18.107396 17606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:18.107504 17607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:18.107512 17611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:18.107939 17601 server_base.cc:1034] running on GCE node
I20260808 10:42:18.108374 17601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:18.108948 17601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:18.110110 17601 hybrid_clock.cc:648] HybridClock initialized: now 1786185738110086 us; error 42 us; skew 500 ppm
Aug 08 10:42:18 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:18.112748 17601 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:18.113721 17601 webserver.cc:460] Webserver started at http://127.1.194.67:41171/ using document root <none> and password file <none>
I20260808 10:42:18.114296 17601 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:18.114413 17601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:18.118330 17601 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 10:42:18.120314 17618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:18.121707 17601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:18.121829 17601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "a10bf3b095bc45c7864ab598ae9398e9"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "d55ddebf2a483d97ec5fb15fbb823ed7"
server_key_iv: "32f7f060f5317e2115bb807e326522b0"
server_key_version: "encryptionkey@0"
I20260808 10:42:18.122417 17601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:18.147326 17601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:18.150348 17601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:18.150559 17601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:18.151170 17601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:18.152020 17601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:18.152100 17601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:18.152177 17601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:18.152230 17601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:18.161541 17601 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:45111
I20260808 10:42:18.161562 17736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:45111 every 8 connection(s)
I20260808 10:42:18.162559 17601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:18.169332 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17601
Aug 08 10:42:18 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:18.176532 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.164464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:38143 (local address 127.1.194.126:34035)
0808 10:42:18.164635 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.164640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.165747 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 10:42:18.166539 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.166542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.166588 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.166663 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:18.167791 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.168303 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.169686 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.170024 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.170137 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.172778 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:18.172786 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:18.172788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:18.172809 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:18.174476 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:18.174999 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:18.175003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:18.175005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:18.175098 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:18.175454 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:18.175458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:18.175460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:18.175636 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:18.175743 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.176238 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.176358 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":3840}
I20260808 10:42:18.177407 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.164866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:38143)
0808 10:42:18.165601 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.165642 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.166368 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.166825 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:18.166831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:18.167189 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:18.167665 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.167678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.168429 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.168431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.169551 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.169573 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.170195 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.170202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.170417 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.171122 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:18.171138 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:18.172610 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:18.174609 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:18.174615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:18.174630 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:18.174891 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:18.175193 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:18.175197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:18.175200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:18.175353 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:18.175744 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:18.175751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:18.175992 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:42:18.176181 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 10:42:18.176500 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":115,"threads_started":1}
I20260808 10:42:18.177850 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.170128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.1.194.126:34035)
0808 10:42:18.170366 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.170370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.170517 (+ 147us) server_negotiation.cc:407] Connection header received
0808 10:42:18.170610 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.170615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.170671 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.170788 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:18.171534 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.172037 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.172938 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.173169 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.173278 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.174727 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:18.174731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:18.174733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:18.174753 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:18.176205 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:18.176628 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:18.176633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:18.176635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:18.176715 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:18.177014 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:18.177019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:18.177021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:18.177184 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:18.177397 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.177562 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.177674 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260808 10:42:18.178663 17737 heartbeater.cc:347] Connected to a master server at 127.1.194.126:34035
I20260808 10:42:18.178931 17737 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:18.179488 17737 heartbeater.cc:511] Master 127.1.194.126:34035 requested a full tablet report, sending...
I20260808 10:42:18.180491 17249 ts_manager.cc:194] Registered new tserver with Master: a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111)
I20260808 10:42:18.181061 17249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:38143
I20260808 10:42:18.191470 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:42:18.195839 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.192105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.1.194.126:34035)
0808 10:42:18.192252 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.192257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.192500 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:42:18.192671 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.192673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.192774 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.192861 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:18.193107 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.193686 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.194506 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.194724 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.194800 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.195077 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:18.195459 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:42:18.195546 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.195611 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.195660 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:42:18.196453 17249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:42:18.199683 17249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:42:18.207319 17379 tablet_service.cc:1467] Processing CreateTablet for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea (DEFAULT_TABLE table=test-table [id=8aed73a08c9443159387abd490b1af7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:18.207989 17379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:18.210678 17337 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Bootstrap starting.
I20260808 10:42:18.211620 17337 tablet_bootstrap.cc:654] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:18.212455 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.206847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.1.194.66:35853)
0808 10:42:18.207259 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.207270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.207292 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:42:18.207368 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.207373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.207545 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.207695 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:18.207948 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.209148 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.211032 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.211859 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.211983 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.212138 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.212239 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.212328 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":101,"threads_started":1}
I20260808 10:42:18.212572 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.206247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45111 (local address 127.0.0.1:44060)
0808 10:42:18.206375 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.206395 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.206478 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.207482 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:18.207485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:18.207500 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:18.207609 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.207616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.209159 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.209164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.210664 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.210675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.212144 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.212149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.212253 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.212292 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:42:18.212347 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:42:18.212430 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":81}
I20260808 10:42:18.212437 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.206695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35853 (local address 127.0.0.1:44818)
0808 10:42:18.207023 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.207038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.207132 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.207746 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:18.207750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:18.207768 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:18.207839 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.207844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.209287 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.209290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.210899 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.210910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.211980 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.211984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.212070 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.212128 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 10:42:18.212210 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:42:18.212292 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":82,"thread_start_us":97,"threads_started":1}
I20260808 10:42:18.213003 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.206492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.1.194.67:45111)
0808 10:42:18.206993 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.207005 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.207029 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:42:18.207117 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.207123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.207339 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.207501 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:18.207708 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.209022 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.210980 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.211973 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.212525 (+ 552us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.212674 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.212730 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.212824 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":193,"threads_started":1}
I20260808 10:42:18.213471 17337 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: No bootstrap required, opened a new log
I20260808 10:42:18.213618 17337 ts_tablet_manager.cc:1397] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:42:18.214080 17337 raft_consensus.cc:348] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.214212 17337 raft_consensus.cc:374] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:18.214249 17337 raft_consensus.cc:729] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a33c83603c5482f883ba856b670357d, State: Initialized, Role: FOLLOWER
I20260808 10:42:18.214366 17337 consensus_queue.cc:260] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.214607 17337 ts_tablet_manager.cc:1428] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:42:18.215915 17527 tablet_service.cc:1467] Processing CreateTablet for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea (DEFAULT_TABLE table=test-table [id=8aed73a08c9443159387abd490b1af7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:18.215907 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea (DEFAULT_TABLE table=test-table [id=8aed73a08c9443159387abd490b1af7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:18.216935 17527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:18.216953 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:18.223654 17751 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Bootstrap starting.
I20260808 10:42:18.225880 17752 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Bootstrap starting.
I20260808 10:42:18.226229 17751 tablet_bootstrap.cc:654] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:18.227104 17751 log.cc:824] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:18.228394 17752 tablet_bootstrap.cc:654] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:18.229239 17752 log.cc:824] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:18.229331 17751 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: No bootstrap required, opened a new log
I20260808 10:42:18.229534 17751 ts_tablet_manager.cc:1397] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:42:18.231282 17752 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: No bootstrap required, opened a new log
I20260808 10:42:18.231468 17752 ts_tablet_manager.cc:1397] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:42:18.233443 17751 raft_consensus.cc:348] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.233750 17751 raft_consensus.cc:374] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:18.233811 17751 raft_consensus.cc:729] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a10bf3b095bc45c7864ab598ae9398e9, State: Initialized, Role: FOLLOWER
I20260808 10:42:18.234261 17751 consensus_queue.cc:260] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.234918 17737 heartbeater.cc:503] Master 127.1.194.126:34035 was elected leader, sending a full tablet report...
I20260808 10:42:18.235289 17751 ts_tablet_manager.cc:1428] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:42:18.235210 17752 raft_consensus.cc:348] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.235460 17752 raft_consensus.cc:374] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:18.235507 17752 raft_consensus.cc:729] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6e504ba3739481c8685a3c393d36d2e, State: Initialized, Role: FOLLOWER
I20260808 10:42:18.235915 17752 consensus_queue.cc:260] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.236541 17598 heartbeater.cc:503] Master 127.1.194.126:34035 was elected leader, sending a full tablet report...
I20260808 10:42:18.236855 17752 ts_tablet_manager.cc:1428] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 10:42:18.258023 17449 tablet.cc:2369] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:42:18.413739 17738 tablet.cc:2369] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:42:18.417470 17756 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:18.417747 17756 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.418883 17756 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539)
I20260808 10:42:18.422936 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.419239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45111 (local address 127.1.194.66:47383)
0808 10:42:18.419392 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.419410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.419527 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.419898 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:18.419901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:18.419919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:18.420045 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.420051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.420779 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.420784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.421716 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.421725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.422465 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.422469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.422606 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.422647 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:42:18.422724 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:42:18.422800 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":37}
I20260808 10:42:18.422936 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.419330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:47383 (local address 127.1.194.67:45111)
0808 10:42:18.419477 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.419482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.419499 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:42:18.419716 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.419720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.419773 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.419885 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:18.420164 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.420668 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.421838 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.422348 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.422471 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.422526 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.422741 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.422811 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:42:18.423853 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.419553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38539 (local address 127.1.194.66:34747)
0808 10:42:18.419880 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.419895 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.419984 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.420300 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:18.420303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:18.420317 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:18.420436 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.420441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.421171 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.421174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.422148 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:18.422168 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.423444 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.423448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:18.423550 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.423576 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:42:18.423655 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:42:18.423713 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":19,"thread_start_us":119,"threads_started":1}
I20260808 10:42:18.423801 17690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a1122de1e4c0b82f2e63b3b3a40ea" candidate_uuid: "c6e504ba3739481c8685a3c393d36d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a10bf3b095bc45c7864ab598ae9398e9" is_pre_election: true
I20260808 10:42:18.423915 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.419695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:34747 (local address 127.1.194.65:38539)
0808 10:42:18.420038 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.420045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.420078 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:42:18.420130 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.420132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.420179 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.420292 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:18.420546 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.421044 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.422293 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.423015 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.423195 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.423342 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.423683 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.423770 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":130,"threads_started":1}
I20260808 10:42:18.424167 17690 raft_consensus.cc:2454] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6e504ba3739481c8685a3c393d36d2e in term 0.
I20260808 10:42:18.424583 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a1122de1e4c0b82f2e63b3b3a40ea" candidate_uuid: "c6e504ba3739481c8685a3c393d36d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a33c83603c5482f883ba856b670357d" is_pre_election: true
I20260808 10:42:18.424643 17476 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e; no voters:
I20260808 10:42:18.424877 17401 raft_consensus.cc:2454] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6e504ba3739481c8685a3c393d36d2e in term 0.
I20260808 10:42:18.424912 17756 raft_consensus.cc:2781] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:42:18.425012 17756 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:42:18.425074 17756 raft_consensus.cc:3037] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:18.426247 17756 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.426630 17756 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [CANDIDATE]: Term 1 election: Requested vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539)
I20260808 10:42:18.426955 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a1122de1e4c0b82f2e63b3b3a40ea" candidate_uuid: "c6e504ba3739481c8685a3c393d36d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a33c83603c5482f883ba856b670357d"
I20260808 10:42:18.426952 17690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a1122de1e4c0b82f2e63b3b3a40ea" candidate_uuid: "c6e504ba3739481c8685a3c393d36d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a10bf3b095bc45c7864ab598ae9398e9"
I20260808 10:42:18.427085 17401 raft_consensus.cc:3037] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:18.427076 17690 raft_consensus.cc:3037] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:18.428107 17401 raft_consensus.cc:2454] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6e504ba3739481c8685a3c393d36d2e in term 1.
I20260808 10:42:18.428264 17690 raft_consensus.cc:2454] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6e504ba3739481c8685a3c393d36d2e in term 1.
I20260808 10:42:18.428388 17476 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a33c83603c5482f883ba856b670357d, c6e504ba3739481c8685a3c393d36d2e; no voters:
I20260808 10:42:18.428617 17756 raft_consensus.cc:2781] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:18.428828 17756 raft_consensus.cc:686] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 1 LEADER]: Becoming Leader. State: Replica: c6e504ba3739481c8685a3c393d36d2e, State: Running, Role: LEADER
I20260808 10:42:18.429147 17756 consensus_queue.cc:237] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.432153 17250 catalog_manager.cc:5495] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e reported cstate change: term changed from 0 to 1, leader changed from <none> to c6e504ba3739481c8685a3c393d36d2e (127.1.194.66). New cstate: current_term: 1 leader_uuid: "c6e504ba3739481c8685a3c393d36d2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260808 10:42:18.447660 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.444498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.1.194.66:35853)
0808 10:42:18.444628 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:18.444631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:18.444643 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:42:18.444713 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:18.444715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:18.444769 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:18.444858 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:18.445052 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.445630 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.446416 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:18.446630 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:18.446707 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:18.446952 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:18.447348 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:42:18.447433 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:18.447489 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:42:18.447535 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:42:18.453461 17401 raft_consensus.cc:1261] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Refusing update from remote peer c6e504ba3739481c8685a3c393d36d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:18.453467 17690 raft_consensus.cc:1261] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Refusing update from remote peer c6e504ba3739481c8685a3c393d36d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:18.454309 17760 consensus_queue.cc:1035] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:18.454512 17756 consensus_queue.cc:1035] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:18.461042 17764 mvcc.cc:205] Tried to move back new op lower bound from 7316216784697671680 to 7316216784608559104. Current Snapshot: MvccSnapshot[applied={T|T < 7316216784697671680}]
I20260808 10:42:18.463155 17765 mvcc.cc:205] Tried to move back new op lower bound from 7316216784697671680 to 7316216784608559104. Current Snapshot: MvccSnapshot[applied={T|T < 7316216784697671680}]
I20260808 10:42:18.468979 17249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:18.469185 17249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:18.471642 17249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Sending DeleteTablet for 3 replicas of tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:18.472402 17379 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:18 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:50080
I20260808 10:42:18.472399 17527 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:18 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:44818
I20260808 10:42:18.472433 17666 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:18 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:44060
I20260808 10:42:18.472823 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 17291
I20260808 10:42:18.474265 17776 tablet_replica.cc:330] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: stopping tablet replica
I20260808 10:42:18.474555 17776 raft_consensus.cc:2229] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:42:18.474850 17776 raft_consensus.cc:2258] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:18.476116 17776 ts_tablet_manager.cc:1905] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:18.477062 17775 tablet_replica.cc:330] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: stopping tablet replica
I20260808 10:42:18.477414 17775 raft_consensus.cc:2229] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:18.477789 17775 raft_consensus.cc:2258] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:18.479071 17776 ts_tablet_manager.cc:1918] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:18.479164 17776 log.cc:1196] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:18.479382 17775 ts_tablet_manager.cc:1905] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:18.479499 17776 ts_tablet_manager.cc:1939] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P c6e504ba3739481c8685a3c393d36d2e: Deleting consensus metadata
I20260808 10:42:18.480759 17236 catalog_manager.cc:4841] TS c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea (table test-table [id=8aed73a08c9443159387abd490b1af7c]) successfully deleted
I20260808 10:42:18.481925 17775 ts_tablet_manager.cc:1918] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:18.482059 17775 log.cc:1196] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:18.482493 17775 ts_tablet_manager.cc:1939] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P a10bf3b095bc45c7864ab598ae9398e9: Deleting consensus metadata
W20260808 10:42:18.482833 17235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:38539 (error 108)
I20260808 10:42:18.484009 17235 catalog_manager.cc:4841] TS a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea (table test-table [id=8aed73a08c9443159387abd490b1af7c]) successfully deleted
W20260808 10:42:18.484427 17748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:42:18.483860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:38539 (local address 127.0.0.1:50082)
0808 10:42:18.483993 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.484007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.484081 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.484307 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:38539: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:42:18.484628 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 17452
W20260808 10:42:18.486783 17235 catalog_manager.cc:4584] TS 0a33c83603c5482f883ba856b670357d (127.1.194.65:38539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e8a1122de1e4c0b82f2e63b3b3a40ea: Network error: Client connection negotiation failed: client connection to 127.1.194.65:38539: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:42:18.492300 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 17601
I20260808 10:42:18.498739 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 17216
I20260808 10:42:18.506153 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34035
--webserver_interface=127.1.194.126
--webserver_port=43975
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:18.618450 17778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:18.618777 17778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:18.618870 17778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:18.623472 17778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:18.623605 17778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:18.623652 17778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:18.623723 17778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:18.623767 17778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:18.629873 17778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:34035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:34035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=43975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:18.631131 17778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:18.632217 17778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:18.641218 17783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:18.641201 17786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:18.641203 17784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:18.641824 17778 server_base.cc:1034] running on GCE node
I20260808 10:42:18.642431 17778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:18.643594 17778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:18.644801 17778 hybrid_clock.cc:648] HybridClock initialized: now 1786185738644780 us; error 32 us; skew 500 ppm
Aug 08 10:42:18 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:18.647928 17778 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:18.648900 17778 webserver.cc:460] Webserver started at http://127.1.194.126:43975/ using document root <none> and password file <none>
I20260808 10:42:18.649526 17778 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:18.649637 17778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:18.653584 17778 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:18.655732 17793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:18.656947 17778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:18.657102 17778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "345cf8c2698d4d6dbc77dc26b444d6f4"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "7eff2c7c0e1ce3bbf1d58bff019954ce"
server_key_iv: "b2ccb0d076613c60be19461796ed8252"
server_key_version: "encryptionkey@0"
I20260808 10:42:18.657714 17778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:18.670496 17778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:18.673759 17778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:18.673959 17778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:18.680999 17778 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:34035
I20260808 10:42:18.681314 17846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:34035 every 8 connection(s)
I20260808 10:42:18.682180 17778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:18.683099 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17778
I20260808 10:42:18.683492 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:38539
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:34035
--webserver_port=34947
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:18.687206 17847 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:42:18.688915 17847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap starting.
I20260808 10:42:18.706246 17847 log.cc:824] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:18.717537 17847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:18.717983 17847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4: Bootstrap complete.
I20260808 10:42:18.722090 17847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:18.722517 17847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345cf8c2698d4d6dbc77dc26b444d6f4, State: Initialized, Role: FOLLOWER
I20260808 10:42:18.722993 17847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:18.723138 17847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:18.723227 17847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:18.723347 17847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:42:18.725216 17847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:18.725620 17847 leader_election.cc:302] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345cf8c2698d4d6dbc77dc26b444d6f4; no voters:
I20260808 10:42:18.725967 17847 leader_election.cc:288] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:42:18.726055 17851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:42:18.726312 17851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [term 3 LEADER]: Becoming Leader. State: Replica: 345cf8c2698d4d6dbc77dc26b444d6f4, State: Running, Role: LEADER
I20260808 10:42:18.726603 17851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } }
I20260808 10:42:18.727232 17847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:18.727596 17851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345cf8c2698d4d6dbc77dc26b444d6f4. Latest consensus state: current_term: 3 leader_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } } }
I20260808 10:42:18.727716 17851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:18.728101 17855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:18.728144 17852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345cf8c2698d4d6dbc77dc26b444d6f4" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 34035 } } }
I20260808 10:42:18.728370 17852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345cf8c2698d4d6dbc77dc26b444d6f4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:18.734589 17855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:18.735414 17855 catalog_manager.cc:1259] Loaded cluster ID: 4433cb84d78449f18f8ec584d7042ea8
I20260808 10:42:18.735502 17855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 10:42:18.736310 17855 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0efe5a09ad google::LogMessage::Fail() at ??:0
@ 0x7f0efe5a388a google::LogMessage::SendToLog() at ??:0
@ 0x7f0efe5a04de google::LogMessage::Flush() at ??:0
@ 0x7f0efe5a1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0f08ee3fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f0f08ee4d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0f08ee1579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f0f08f25640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0f007f23d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0f007f2c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f0f007f43f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0f007e3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0f01e276db start_thread at ??:0
@ 0x7f0efd09e61f clone at ??:0
W20260808 10:42:18.831166 17849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:18.831593 17849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:18.831745 17849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:18.836040 17849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:18.836110 17849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:18.836174 17849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:18.841812 17849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:38539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=34947
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:18.842962 17849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:18.844008 17849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:18.855033 17892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:18.855034 17891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:18.855378 17849 server_base.cc:1034] running on GCE node
W20260808 10:42:18.855473 17894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:18.855909 17849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:18.856663 17849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:18.857861 17849 hybrid_clock.cc:648] HybridClock initialized: now 1786185738857840 us; error 44 us; skew 500 ppm
Aug 08 10:42:18 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:18.861064 17849 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:18.862213 17849 webserver.cc:460] Webserver started at http://127.1.194.65:34947/ using document root <none> and password file <none>
I20260808 10:42:18.862825 17849 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:18.862919 17849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:18.867499 17849 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:18.870021 17902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:18.871482 17849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:18.871702 17849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "0a33c83603c5482f883ba856b670357d"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "eaff58c194d32bb861878776e62feccf"
server_key_iv: "7c5c3a5bd86855598cb95d896da6fb3d"
server_key_version: "encryptionkey@0"
I20260808 10:42:18.872298 17849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:18.894397 17849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:18.898101 17849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:18.898321 17849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:18.898953 17849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:18.900108 17911 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:42:18.905972 17849 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:42:18.906106 17849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 10:42:18.906203 17849 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:42:18.908093 17849 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:42:18.908180 17849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:42:18.908488 17911 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Bootstrap starting.
I20260808 10:42:18.919252 17849 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:38539
I20260808 10:42:18.919323 18026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:38539 every 8 connection(s)
I20260808 10:42:18.920305 17849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:18.922173 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 17849
I20260808 10:42:18.922654 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:35853
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:34035
--webserver_port=40427
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:18.929943 17911 log.cc:824] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:18.937115 17911 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:18.937611 17911 tablet_bootstrap.cc:492] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Bootstrap complete.
I20260808 10:42:18.938752 17911 ts_tablet_manager.cc:1397] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.014s
I20260808 10:42:18.941219 17911 raft_consensus.cc:348] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.941752 17911 raft_consensus.cc:729] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a33c83603c5482f883ba856b670357d, State: Initialized, Role: FOLLOWER
I20260808 10:42:18.942274 17911 consensus_queue.cc:260] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:18.943251 17911 ts_tablet_manager.cc:1428] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
W20260808 10:42:18.953214 18029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:42:18.922552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:34519)
0808 10:42:18.923196 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.923265 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:18.924406 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:18.952385 (+ 27979us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":167,"threads_started":1}
W20260808 10:42:18.953883 18027 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:34035 (0 consecutive failures): Network error: Failed to ping master at 127.1.194.126:34035: Client connection negotiation failed: client connection to 127.1.194.126:34035: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:42:18.954656 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.954386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:51705)
0808 10:42:18.954516 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.954596 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:42:18.955376 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:18.955167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:53487)
0808 10:42:18.955280 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:18.955332 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:42:18.955569 18027 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:42:19.049280 18030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:19.049752 18030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:19.049847 18030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:19.054177 18030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:19.054273 18030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:19.054364 18030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:19.059854 18030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:35853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=40427
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:19.060990 18030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:19.062091 18030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:19.072808 18037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:19.072939 18030 server_base.cc:1034] running on GCE node
W20260808 10:42:19.072855 18038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:19.072808 18040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:19.073668 18030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:19.074421 18030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:19.075642 18030 hybrid_clock.cc:648] HybridClock initialized: now 1786185739075623 us; error 36 us; skew 500 ppm
Aug 08 10:42:19 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185739, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:19.079102 18030 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:19.080080 18030 webserver.cc:460] Webserver started at http://127.1.194.66:40427/ using document root <none> and password file <none>
I20260808 10:42:19.080672 18030 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:19.080770 18030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:19.085098 18030 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:42:19.087267 18050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:19.088510 18030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:19.088703 18030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "c6e504ba3739481c8685a3c393d36d2e"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "c57807c7956057346917e830bee3c3dd"
server_key_iv: "cbcb788ba3df8973de31bd5eaa723104"
server_key_version: "encryptionkey@0"
I20260808 10:42:19.089290 18030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:19.102495 18030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:19.105913 18030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:19.106159 18030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:19.106845 18030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:19.107810 18030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:19.107882 18030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:19.107955 18030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:19.107999 18030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:19.117410 18030 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:35853
I20260808 10:42:19.117448 18171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:35853 every 8 connection(s)
I20260808 10:42:19.118523 18030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:19.122607 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.120991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:46425)
0808 10:42:19.121756 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.121862 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":119,"threads_started":1}
W20260808 10:42:19.123062 18172 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:34035 (0 consecutive failures): Network error: Failed to ping master at 127.1.194.126:34035: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
I20260808 10:42:19.123670 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.123472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:51149)
0808 10:42:19.123572 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.123621 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 10:42:19.124284 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.124066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:40149)
0808 10:42:19.124185 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.124231 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 10:42:19.124490 18172 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:42:19.125023 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18030
I20260808 10:42:19.125451 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:45111
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:34035
--webserver_port=41171
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:44409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:19.236958 18175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:19.237241 18175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:19.237285 18175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:19.241287 18175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:19.241355 18175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:19.241487 18175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:19.246867 18175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:44409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:45111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=41171
--tserver_master_addrs=127.1.194.126:34035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:19.247876 18175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:19.248827 18175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:19.259202 18181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:19.259220 18182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:19.259361 18175 server_base.cc:1034] running on GCE node
W20260808 10:42:19.259204 18185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:19.259908 18175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:19.260555 18175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:19.261772 18175 hybrid_clock.cc:648] HybridClock initialized: now 1786185739261735 us; error 46 us; skew 500 ppm
Aug 08 10:42:19 dist-test-slave-ss4n krb5kdc[16634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185739, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:19.264544 18175 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:19.265426 18175 webserver.cc:460] Webserver started at http://127.1.194.67:41171/ using document root <none> and password file <none>
I20260808 10:42:19.266031 18175 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:19.266156 18175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:19.270243 18175 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:19.272361 18192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:19.273855 18175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:42:19.274010 18175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "a10bf3b095bc45c7864ab598ae9398e9"
format_stamp: "Formatted at 2026-08-08 10:42:16 on dist-test-slave-ss4n"
server_key: "d55ddebf2a483d97ec5fb15fbb823ed7"
server_key_iv: "32f7f060f5317e2115bb807e326522b0"
server_key_version: "encryptionkey@0"
I20260808 10:42:19.274626 18175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:19.296198 18175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:19.299719 18175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:19.299938 18175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:19.300582 18175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:19.301445 18175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:19.301568 18175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:19.301684 18175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:19.301728 18175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:19.311286 18175 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:45111
I20260808 10:42:19.311307 18312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:45111 every 8 connection(s)
I20260808 10:42:19.312348 18175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:19.312827 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18175
I20260808 10:42:19.316250 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.314759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:60391)
0808 10:42:19.315423 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.315524 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":112,"threads_started":1}
W20260808 10:42:19.316803 18313 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:34035 (0 consecutive failures): Network error: Failed to ping master at 127.1.194.126:34035: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
I20260808 10:42:19.317412 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.317202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:45161)
0808 10:42:19.317306 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.317357 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 10:42:19.318038 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.317848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:34183)
0808 10:42:19.317942 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.317983 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 10:42:19.318276 18313 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:42:19.956660 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:19.956153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:49085)
0808 10:42:19.956402 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:19.956527 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":72,"threads_started":1}
I20260808 10:42:20.125396 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.125007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:39123)
0808 10:42:20.125244 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:20.125297 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":70,"threads_started":1}
I20260808 10:42:20.319483 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.319030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:41547)
0808 10:42:20.319318 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:20.319381 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260808 10:42:20.332871 18320 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:20.333127 18320 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:20.334312 18320 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
Aug 08 10:42:20 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.66@KRBTEST.COM
Aug 08 10:42:20 dist-test-slave-ss4n krb5kdc[16634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.67@KRBTEST.COM
I20260808 10:42:20.355289 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.334484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45111 (local address 127.1.194.65:33883)
0808 10:42:20.334711 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:20.334730 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:20.334847 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:20.336603 (+ 1756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:20.336609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:20.337188 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:20.337683 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:20.337698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.339433 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.339436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:20.340208 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:20.340217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.341009 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.341013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:20.341180 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:20.348207 (+ 7027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:20.348227 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:20.350543 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:20.352814 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:20.352820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:20.352823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:20.353124 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:20.353506 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:20.353512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:20.353514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:20.353658 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:20.354317 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:20.354324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:20.354565 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:42:20.354739 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 10:42:20.355088 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":89,"spinlock_wait_cycles":9984}
I20260808 10:42:20.355301 18322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.334727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:33883 (local address 127.1.194.67:45111)
0808 10:42:20.335098 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:20.335130 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:20.335181 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:42:20.335929 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:20.335949 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:20.336248 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:20.336554 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:20.337876 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.339259 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.340344 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.340852 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.341054 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:20.350773 (+ 9719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:20.350778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:20.350789 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:20.350819 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:20.352690 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:20.353246 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:20.353253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:20.353260 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:20.353382 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:20.353757 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:20.353762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:20.353764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:20.354200 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:20.354339 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:20.354864 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 10:42:20.355133 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":134,"threads_started":1}
I20260808 10:42:20.355340 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.334626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35853 (local address 127.1.194.65:35885)
0808 10:42:20.335074 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:20.335091 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:20.335183 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:20.336603 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:20.336610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:20.337191 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:20.337684 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:20.337699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.339167 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.339171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:20.340211 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:20.340222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.340971 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.340975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:20.341182 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:20.348311 (+ 7129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:20.348324 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:20.350308 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:20.352733 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:20.352740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:20.352754 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:20.353118 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:20.353551 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:20.353555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:20.353557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:20.353662 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:20.354321 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:20.354323 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:20.354559 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 10:42:20.354740 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 10:42:20.355174 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":235,"spinlock_wait_cycles":29440,"thread_start_us":105,"threads_started":1}
I20260808 10:42:20.355957 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.334817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:35885 (local address 127.1.194.66:35853)
0808 10:42:20.335174 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:20.335194 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:20.335228 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:42:20.335885 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:20.335890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:20.336219 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:20.336515 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:20.337878 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.339031 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.340385 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:20.340840 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:20.341002 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:20.350586 (+ 9584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:20.350591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:20.350604 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:20.350635 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:20.352595 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:20.353253 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:20.353258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:20.353262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:20.353382 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:20.353768 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:20.353771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:20.353775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:20.354200 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:20.354339 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:20.354863 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 10:42:20.355163 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":93,"threads_started":1}
W20260808 10:42:20.357918 17905 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
W20260808 10:42:20.358070 17904 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:20.358206 17904 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a33c83603c5482f883ba856b670357d; no voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e
I20260808 10:42:20.358498 18320 raft_consensus.cc:2726] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:42:20.958022 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:20.957582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:43727)
0808 10:42:20.957866 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:20.957928 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":78,"threads_started":1}
I20260808 10:42:21.126690 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:21.126215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:57575)
0808 10:42:21.126484 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:21.126556 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260808 10:42:21.320832 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:21.320346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:44099)
0808 10:42:21.320658 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:21.320726 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":96,"threads_started":1}
I20260808 10:42:21.948747 18328 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:21.948877 18328 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:21.949270 18328 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
W20260808 10:42:21.950178 17904 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
W20260808 10:42:21.950289 17905 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:21.950402 17905 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a33c83603c5482f883ba856b670357d; no voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e
I20260808 10:42:21.950591 18328 raft_consensus.cc:2726] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:42:21.959122 18329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:21.958757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:45777)
0808 10:42:21.958985 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:21.959037 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260808 10:42:22.127938 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:22.127504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:43627)
0808 10:42:22.127782 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:22.127843 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":84,"threads_started":1}
I20260808 10:42:22.322171 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:22.321679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:50157)
0808 10:42:22.321980 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:22.322048 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":76,"threads_started":1}
I20260808 10:42:22.960409 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:22.959929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:38817)
0808 10:42:22.960236 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:22.960306 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":107,"threads_started":1}
I20260808 10:42:23.129284 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:23.128831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:35479)
0808 10:42:23.129119 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:23.129184 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
I20260808 10:42:23.323458 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:23.322996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:45735)
0808 10:42:23.323290 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:23.323360 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":91,"threads_started":1}
I20260808 10:42:23.603039 18335 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:23.603281 18335 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:23.603796 18335 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
W20260808 10:42:23.604758 17904 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
W20260808 10:42:23.604876 17905 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:23.604985 17905 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a33c83603c5482f883ba856b670357d; no voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e
I20260808 10:42:23.605180 18335 raft_consensus.cc:2726] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:42:23.961970 18336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:23.961455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:59611)
0808 10:42:23.961814 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:23.961871 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":80,"threads_started":1}
I20260808 10:42:24.130555 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:24.130155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:42469)
0808 10:42:24.130397 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:24.130455 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260808 10:42:24.324851 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:24.324313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:48021)
0808 10:42:24.324678 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:24.324750 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":83,"threads_started":1}
I20260808 10:42:24.963354 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:24.962875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:47123)
0808 10:42:24.963160 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:24.963249 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":73,"threads_started":1}
I20260808 10:42:25.131791 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:25.131330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:44775)
0808 10:42:25.131635 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:25.131695 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
I20260808 10:42:25.326283 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:25.325794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:38939)
0808 10:42:25.326121 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:25.326188 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
I20260808 10:42:25.562039 18342 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:25.562196 18342 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:25.562620 18342 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
W20260808 10:42:25.563381 17904 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
W20260808 10:42:25.563572 17905 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:25.563661 17905 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a33c83603c5482f883ba856b670357d; no voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e
I20260808 10:42:25.563820 18342 raft_consensus.cc:2726] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:42:25.964586 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:25.964141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:54713)
0808 10:42:25.964426 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:25.964483 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
I20260808 10:42:26.133090 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:26.132575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:42869)
0808 10:42:26.132908 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:26.132965 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":73,"threads_started":1}
I20260808 10:42:26.327543 18345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:26.327085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:59475)
0808 10:42:26.327391 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:26.327451 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":74,"threads_started":1}
I20260808 10:42:26.965899 18346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:26.965425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:39613)
0808 10:42:26.965741 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:26.965802 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":131,"threads_started":1}
I20260808 10:42:27.134416 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:27.133983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:38257)
0808 10:42:27.134245 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:27.134312 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
I20260808 10:42:27.156070 18348 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:27.156229 18348 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:27.156908 18348 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
W20260808 10:42:27.157541 17905 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
W20260808 10:42:27.157686 17904 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:27.157783 17904 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a33c83603c5482f883ba856b670357d; no voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e
I20260808 10:42:27.157939 18348 raft_consensus.cc:2726] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:42:27.328946 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:27.328512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:57063)
0808 10:42:27.328785 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:27.328851 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260808 10:42:27.967084 18350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:27.966670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:60877)
0808 10:42:27.966923 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:27.966980 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260808 10:42:28.135838 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:28.135332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:34177)
0808 10:42:28.135677 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:28.135740 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":102,"threads_started":1}
I20260808 10:42:28.330230 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:28.329794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:56993)
0808 10:42:28.330060 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:28.330132 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260808 10:42:28.968465 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:28.967998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:34261)
0808 10:42:28.968296 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:28.968359 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":78,"threads_started":1}
I20260808 10:42:29.137169 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:29.136732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:50507)
0808 10:42:29.136989 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:29.137049 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260808 10:42:29.331418 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:29.330976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:48111)
0808 10:42:29.331259 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:29.331319 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":78,"threads_started":1}
I20260808 10:42:29.969794 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:29.969295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:50249)
0808 10:42:29.969633 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:29.969695 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":80,"threads_started":1}
I20260808 10:42:30.138566 18358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:30.138074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:34179)
0808 10:42:30.138346 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:30.138408 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260808 10:42:30.332720 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:30.332234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:33811)
0808 10:42:30.332554 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:30.332620 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":83,"threads_started":1}
I20260808 10:42:30.970966 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:30.970560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:53287)
0808 10:42:30.970816 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:30.970873 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":88,"threads_started":1}
I20260808 10:42:31.139905 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:31.139444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:33515)
0808 10:42:31.139737 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:31.139803 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":79,"threads_started":1}
I20260808 10:42:31.334137 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:31.333685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:47089)
0808 10:42:31.333980 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:31.334039 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":94,"threads_started":1}
I20260808 10:42:31.972311 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:31.971896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:59909)
0808 10:42:31.972160 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:31.972216 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
I20260808 10:42:32.141242 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:32.140818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:43227)
0808 10:42:32.141078 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:32.141141 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260808 10:42:32.243677 18365 raft_consensus.cc:482] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:32.243820 18365 raft_consensus.cc:504] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10bf3b095bc45c7864ab598ae9398e9" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45111 } } peers { permanent_uuid: "c6e504ba3739481c8685a3c393d36d2e" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35853 } } peers { permanent_uuid: "0a33c83603c5482f883ba856b670357d" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 38539 } }
I20260808 10:42:32.244419 18365 leader_election.cc:288] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111), c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853)
W20260808 10:42:32.245420 17905 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6e504ba3739481c8685a3c393d36d2e (127.1.194.66:35853): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
W20260808 10:42:32.245558 17904 leader_election.cc:341] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a10bf3b095bc45c7864ab598ae9398e9 (127.1.194.67:45111): Not found: Tablet not found: 3e8a1122de1e4c0b82f2e63b3b3a40ea
I20260808 10:42:32.245637 17904 leader_election.cc:302] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a33c83603c5482f883ba856b670357d; no voters: a10bf3b095bc45c7864ab598ae9398e9, c6e504ba3739481c8685a3c393d36d2e
I20260808 10:42:32.245826 18365 raft_consensus.cc:2726] T 3e8a1122de1e4c0b82f2e63b3b3a40ea P 0a33c83603c5482f883ba856b670357d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:42:32.335330 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:32.334920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:57957)
0808 10:42:32.335181 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:32.335237 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260808 10:42:32.973658 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:32.973136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:52325)
0808 10:42:32.973437 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:32.973558 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":127,"threads_started":1}
I20260808 10:42:33.142717 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:33.142264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:53441)
0808 10:42:33.142556 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:33.142617 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
I20260808 10:42:33.336701 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:33.336241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.67:55025)
0808 10:42:33.336529 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:33.336606 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":123,"threads_started":1}
I20260808 10:42:33.975040 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:33.974618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.65:57987)
0808 10:42:33.974886 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:33.974944 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
I20260808 10:42:34.144212 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:34.143749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:34035 (local address 127.1.194.66:40693)
0808 10:42:34.144058 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:34.144110 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":117,"threads_started":1}
I20260808 10:42:34.319257 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 17849
I20260808 10:42:34.326987 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18030
I20260808 10:42:34.333527 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18175
2026-08-08T10:42:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19549 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:42:34 dist-test-slave-ss4n krb5kdc[18375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:42:34 dist-test-slave-ss4n krb5kdc[18375](info): set up 2 sockets
Aug 08 10:42:34 dist-test-slave-ss4n krb5kdc[18375](info): commencing operation
krb5kdc: starting...
W20260808 10:42:35.457855 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:42:35 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:42:35.593578 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37735
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:37735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:35.709642 18388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:35.709931 18388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:35.709973 18388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:35.714061 18388 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:42:35.714126 18388 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:42:35.714143 18388 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:42:35.714156 18388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:35.714175 18388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:35.714195 18388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:35.714210 18388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:35.714224 18388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:35.719573 18388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:37735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:35.720692 18388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:35.721783 18388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:35.730581 18393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:35.730531 18394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:35.730531 18396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:35.730870 18388 server_base.cc:1034] running on GCE node
I20260808 10:42:35.731456 18388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:35.732618 18388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:35.733862 18388 hybrid_clock.cc:648] HybridClock initialized: now 1786185755733828 us; error 46 us; skew 500 ppm
Aug 08 10:42:35 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185755, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:35.736865 18388 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:35.737855 18388 webserver.cc:460] Webserver started at http://127.1.194.126:38473/ using document root <none> and password file <none>
I20260808 10:42:35.738436 18388 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:35.738533 18388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:35.738765 18388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:35.740841 18388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "f084c21f2bf04dd98f1eac75595c9760"
format_stamp: "Formatted at 2026-08-08 10:42:35 on dist-test-slave-ss4n"
server_key: "835d74febcff69bf2b5de021063f661f"
server_key_iv: "43194476d2ea256a3eeb65598d543095"
server_key_version: "encryptionkey@0"
I20260808 10:42:35.741434 18388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "f084c21f2bf04dd98f1eac75595c9760"
format_stamp: "Formatted at 2026-08-08 10:42:35 on dist-test-slave-ss4n"
server_key: "835d74febcff69bf2b5de021063f661f"
server_key_iv: "43194476d2ea256a3eeb65598d543095"
server_key_version: "encryptionkey@0"
I20260808 10:42:35.745129 18388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 10:42:35.747556 18405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:35.748785 18388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:35.748929 18388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "f084c21f2bf04dd98f1eac75595c9760"
format_stamp: "Formatted at 2026-08-08 10:42:35 on dist-test-slave-ss4n"
server_key: "835d74febcff69bf2b5de021063f661f"
server_key_iv: "43194476d2ea256a3eeb65598d543095"
server_key_version: "encryptionkey@0"
I20260808 10:42:35.749044 18388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:35.763447 18388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:35.766850 18388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:35.767076 18388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:35.774335 18388 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:37735
I20260808 10:42:35.774727 18462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:37735 every 8 connection(s)
I20260808 10:42:35.775323 18388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:35.778563 18463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:35.780997 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18388
I20260808 10:42:35.781167 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:42:35.781451 1801 external_mini_cluster.cc:1428] Setting key a9775ed496d543950177ca0b2c154c35
I20260808 10:42:35.784894 18463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap starting.
I20260808 10:42:35.787252 18463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:35.788058 18463 log.cc:824] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:35.790287 18463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: No bootstrap required, opened a new log
Aug 08 10:42:35 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:35.795328 18463 raft_consensus.cc:348] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:35.795624 18463 raft_consensus.cc:374] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:35.795681 18463 raft_consensus.cc:729] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f084c21f2bf04dd98f1eac75595c9760, State: Initialized, Role: FOLLOWER
I20260808 10:42:35.796185 18463 consensus_queue.cc:260] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:35.796352 18463 raft_consensus.cc:388] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:35.796429 18463 raft_consensus.cc:482] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:35.796530 18463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:35.796700 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:35.782947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.1.194.126:37735)
0808 10:42:35.783396 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:35.783410 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:35.783445 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:42:35.784125 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:35.784141 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:35.784456 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:35.784800 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:35.785696 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:35.786674 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:35.787604 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:35.788066 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:35.788250 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:35.791043 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:35.791050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:35.791066 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:35.791100 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:35.793333 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:35.793842 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:35.793848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:35.793855 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:35.794020 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:35.794322 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:35.794325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:35.794327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:35.794893 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:35.795111 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:35.795330 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:42:35.795676 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":141,"threads_started":1}
I20260808 10:42:35.797792 18463 raft_consensus.cc:504] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:35.798185 18463 leader_election.cc:302] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f084c21f2bf04dd98f1eac75595c9760; no voters:
I20260808 10:42:35.798563 18463 leader_election.cc:288] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:42:35.798662 18468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:35.798877 18468 raft_consensus.cc:686] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 LEADER]: Becoming Leader. State: Replica: f084c21f2bf04dd98f1eac75595c9760, State: Running, Role: LEADER
I20260808 10:42:35.799281 18468 consensus_queue.cc:237] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:35.799932 18463 sys_catalog.cc:564] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:35.801453 18470 sys_catalog.cc:455] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f084c21f2bf04dd98f1eac75595c9760. Latest consensus state: current_term: 1 leader_uuid: "f084c21f2bf04dd98f1eac75595c9760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } } }
I20260808 10:42:35.801635 18469 sys_catalog.cc:455] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f084c21f2bf04dd98f1eac75595c9760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } } }
I20260808 10:42:35.801687 18470 sys_catalog.cc:458] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:35.801710 18469 sys_catalog.cc:458] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:35.802152 18479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:35.805348 18479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:35.811080 18479 catalog_manager.cc:1347] Generated new cluster ID: a2bba6a0ea7545819590b809fccb26d6
I20260808 10:42:35.811160 18479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:35.818061 18479 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:42:35.819120 18479 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:35.825348 18479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Generated new TSK 0
I20260808 10:42:35.826077 18479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:42:35.861750 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37735
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:35.978119 18489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:35.978367 18489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:35.978410 18489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:35.982352 18489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:35.982407 18489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:35.982465 18489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:35.987526 18489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:35.988605 18489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:35.989630 18489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:35.999648 18498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:35.999764 18495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:35.999656 18496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:35.999948 18489 server_base.cc:1034] running on GCE node
I20260808 10:42:36.000511 18489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:36.001282 18489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:36.002526 18489 hybrid_clock.cc:648] HybridClock initialized: now 1786185756002496 us; error 54 us; skew 500 ppm
Aug 08 10:42:35 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185755, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:36.005455 18489 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:36.006556 18489 webserver.cc:460] Webserver started at http://127.1.194.65:39305/ using document root <none> and password file <none>
I20260808 10:42:36.007165 18489 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:36.007268 18489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:36.007500 18489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:36.009673 18489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "e0534a64ea264cfbac23bea2636b8a24"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "dcd307ccd9a5e89d0691585270d3b5f9"
server_key_iv: "c7e8f33e8f9afe766ca662a14dab588f"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.010284 18489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "e0534a64ea264cfbac23bea2636b8a24"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "dcd307ccd9a5e89d0691585270d3b5f9"
server_key_iv: "c7e8f33e8f9afe766ca662a14dab588f"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.014122 18489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:42:36.016708 18505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.018059 18489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:36.018195 18489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "e0534a64ea264cfbac23bea2636b8a24"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "dcd307ccd9a5e89d0691585270d3b5f9"
server_key_iv: "c7e8f33e8f9afe766ca662a14dab588f"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.018325 18489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:36.031513 18489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:36.034886 18489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:36.035043 18489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:36.035566 18489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:36.036532 18489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:36.036588 18489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.036629 18489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:36.036645 18489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.046818 18489 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:37473
I20260808 10:42:36.046924 18624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:37473 every 8 connection(s)
I20260808 10:42:36.047775 18489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:36.048976 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18489
I20260808 10:42:36.049126 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:42:36.049479 1801 external_mini_cluster.cc:1428] Setting key f6f92de6f38fc2b72cbb72785af99fd3
Aug 08 10:42:36 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185755, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:42:36.064185 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.050058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:33163 (local address 127.1.194.126:37735)
0808 10:42:36.050181 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.050185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.051288 (+ 1103us) server_negotiation.cc:407] Connection header received
0808 10:42:36.053237 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.053240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.053317 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.053420 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:36.054746 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.055316 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.056597 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.056856 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.056976 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.059954 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:36.059959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:36.059962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:36.059996 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:36.061759 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.062335 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.062338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.062339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.062467 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.062879 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.062886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.062891 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.063116 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:36.063204 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.063912 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.064051 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:42:36.065261 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.050521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:33163)
0808 10:42:36.051110 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.051170 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.052935 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.053724 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.053732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.054099 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:36.054588 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.054602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.055452 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.055456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.056451 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.056468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.057025 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.057029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.057209 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.057938 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:36.057955 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:36.059732 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:36.061893 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:36.061897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:36.061910 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:36.062205 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:36.062581 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:36.062583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:36.062585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:36.062777 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:36.063237 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:36.063247 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:36.063597 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.063823 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.064229 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":161,"threads_started":1}
I20260808 10:42:36.066715 18625 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37735
I20260808 10:42:36.066993 18625 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:36.067513 18625 heartbeater.cc:511] Master 127.1.194.126:37735 requested a full tablet report, sending...
I20260808 10:42:36.069087 18422 ts_manager.cc:194] Registered new tserver with Master: e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473)
I20260808 10:42:36.070294 18422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:33163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:42:36.081750 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37735
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:36.194823 18630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:36.195113 18630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:36.195158 18630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:36.199275 18630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:36.199337 18630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:36.199396 18630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:36.204912 18630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:36.205967 18630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:36.207074 18630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:36.216768 18638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:36.216966 18630 server_base.cc:1034] running on GCE node
W20260808 10:42:36.216768 18635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:36.216848 18636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:36.217715 18630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:36.218319 18630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:36.219607 18630 hybrid_clock.cc:648] HybridClock initialized: now 1786185756219580 us; error 40 us; skew 500 ppm
Aug 08 10:42:36 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:36.222793 18630 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:36.223690 18630 webserver.cc:460] Webserver started at http://127.1.194.66:33149/ using document root <none> and password file <none>
I20260808 10:42:36.224232 18630 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:36.224293 18630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:36.224506 18630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:36.226550 18630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "6d7191491dbf49c28e0470c66c345217"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "286e4af6ac5b58636e509c2e489c39d7"
server_key_iv: "2b0a315f1dafb755102e4a9dd36872f3"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.227100 18630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "6d7191491dbf49c28e0470c66c345217"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "286e4af6ac5b58636e509c2e489c39d7"
server_key_iv: "2b0a315f1dafb755102e4a9dd36872f3"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.230787 18630 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:36.233073 18645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.234326 18630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:42:36.234444 18630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "6d7191491dbf49c28e0470c66c345217"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "286e4af6ac5b58636e509c2e489c39d7"
server_key_iv: "2b0a315f1dafb755102e4a9dd36872f3"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.234570 18630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:36.248984 18630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:36.251827 18630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:36.252027 18630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:36.252642 18630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:36.253420 18630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:36.253464 18630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.253564 18630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:36.253620 18630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.262424 18630 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:34851
I20260808 10:42:36.262454 18762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:34851 every 8 connection(s)
I20260808 10:42:36.263373 18630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:36.269518 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18630
I20260808 10:42:36.269670 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:42:36.269987 1801 external_mini_cluster.cc:1428] Setting key 024460dc86717249447ab60462b613fd
Aug 08 10:42:36 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:36.279817 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.265182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:51901 (local address 127.1.194.126:37735)
0808 10:42:36.265358 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.265364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.266349 (+ 985us) server_negotiation.cc:407] Connection header received
0808 10:42:36.267699 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.267701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.267749 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.267823 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:36.269310 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.270075 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.271674 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.271888 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.271967 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.275557 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:36.275561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:36.275563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:36.275596 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:36.277292 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.278015 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.278018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.278020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.278100 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.278588 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.278590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.278592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.278774 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:36.278853 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.279595 (+ 742us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.279700 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:42:36.281155 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.265653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:51901)
0808 10:42:36.266178 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.266225 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.267487 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.268041 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.268048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.268443 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:36.269113 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.269129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.270203 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.270207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.271542 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.271555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.272069 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.272077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.272330 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.273010 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:36.273038 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:36.275296 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:36.277419 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:36.277426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:36.277444 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:36.277898 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:36.278209 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:36.278216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:36.278220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:36.278438 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:36.278894 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:36.278904 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:36.279298 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.279538 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.279956 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:42:36.282550 18763 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37735
I20260808 10:42:36.282781 18763 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:36.283363 18763 heartbeater.cc:511] Master 127.1.194.126:37735 requested a full tablet report, sending...
I20260808 10:42:36.284457 18422 ts_manager.cc:194] Registered new tserver with Master: 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
I20260808 10:42:36.285291 18422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:51901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:42:36.305408 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37735
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:36.423444 18768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:36.423708 18768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:36.423782 18768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:36.427891 18768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:36.427955 18768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:36.428015 18768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:36.433357 18768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:36.434477 18768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:36.435540 18768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:36.445875 18777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:36.445957 18775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:36.445902 18774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:36.446130 18768 server_base.cc:1034] running on GCE node
I20260808 10:42:36.446549 18768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:36.447204 18768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:36.448405 18768 hybrid_clock.cc:648] HybridClock initialized: now 1786185756448392 us; error 28 us; skew 500 ppm
Aug 08 10:42:36 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:36.451332 18768 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:36.452349 18768 webserver.cc:460] Webserver started at http://127.1.194.67:37963/ using document root <none> and password file <none>
I20260808 10:42:36.452987 18768 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:36.453078 18768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:36.453326 18768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:42:36.455673 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "4027456275494db3a5aa5323b1e40c09"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "c51f59f8bf02fd8cec90f352fc0abbeb"
server_key_iv: "91b599424d360af1b1f2eacf3bb36df8"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.456489 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "4027456275494db3a5aa5323b1e40c09"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "c51f59f8bf02fd8cec90f352fc0abbeb"
server_key_iv: "91b599424d360af1b1f2eacf3bb36df8"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.460783 18768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:42:36.463439 18785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.464740 18768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:42:36.464883 18768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "4027456275494db3a5aa5323b1e40c09"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "c51f59f8bf02fd8cec90f352fc0abbeb"
server_key_iv: "91b599424d360af1b1f2eacf3bb36df8"
server_key_version: "encryptionkey@0"
I20260808 10:42:36.465005 18768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:36.477592 18768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:36.480703 18768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:36.480908 18768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:36.481554 18768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:36.482385 18768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:36.482460 18768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.482529 18768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:36.482577 18768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:36.491531 18768 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:44835
I20260808 10:42:36.491600 18907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:44835 every 8 connection(s)
I20260808 10:42:36.492424 18768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:36.493161 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18768
I20260808 10:42:36.493324 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:42:36.493767 1801 external_mini_cluster.cc:1428] Setting key ef3573d29528d7a6c6bad978d62091c1
Aug 08 10:42:36 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:36.507994 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.494583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:43771 (local address 127.1.194.126:37735)
0808 10:42:36.494738 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.494741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.495820 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 10:42:36.497306 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.497309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.497368 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.497470 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:36.498715 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.499314 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.500772 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.501026 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.501156 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.503914 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:36.503917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:36.503919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:36.503944 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:36.505558 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.506009 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.506012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.506013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.506090 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.506610 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.506613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.506616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.506865 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:36.506966 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.507700 (+ 734us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.507818 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:42:36.508910 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.495111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:43771)
0808 10:42:36.495665 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.495721 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.497082 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.497663 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.497677 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.498064 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:36.498557 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.498572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.499453 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.499458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.500518 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.500529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.501217 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.501220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.501386 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.502003 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:36.502019 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:36.503747 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:36.505659 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:36.505664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:36.505675 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:36.505917 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:36.506232 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:36.506235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:36.506238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:36.506454 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:36.507002 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:36.507035 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:36.507387 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.507655 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.508021 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":168,"threads_started":1}
I20260808 10:42:36.510243 18908 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37735
I20260808 10:42:36.510502 18908 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:36.511063 18908 heartbeater.cc:511] Master 127.1.194.126:37735 requested a full tablet report, sending...
I20260808 10:42:36.512298 18422 ts_manager.cc:194] Registered new tserver with Master: 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835)
I20260808 10:42:36.513018 18422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:43771
I20260808 10:42:36.520269 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:42:36.530202 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.522525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39696 (local address 127.1.194.126:37735)
0808 10:42:36.522685 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.522689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.522878 (+ 189us) server_negotiation.cc:407] Connection header received
0808 10:42:36.522966 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.522968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.523067 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.523203 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:36.523876 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.524396 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.525210 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.525425 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.525632 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.526883 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:36.526885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:36.526887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:36.526910 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:36.528404 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.528794 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.528797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.528798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.528872 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:36.529211 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:36.529214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:36.529215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:36.529468 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:36.529751 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.529968 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.530077 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:42:36.533210 18422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:42:36.536491 18422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:42:36.552114 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.545737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:34851 (local address 127.0.0.1:52042)
0808 10:42:36.546745 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.546758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.546834 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.547407 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.547412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.547433 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:36.547601 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.547610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.549128 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.549131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.550735 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.550744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.551547 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.551550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.551654 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.551690 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.551804 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.551932 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":82,"thread_start_us":96,"threads_started":1}
I20260808 10:42:36.552155 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.546493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.1.194.66:34851)
0808 10:42:36.546939 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.546951 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.546976 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:42:36.547058 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.547063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.547270 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.547421 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:36.547738 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.548983 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.550859 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.551409 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.551534 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.551697 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.551843 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.551993 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":142,"threads_started":1}
I20260808 10:42:36.552414 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.545974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44835 (local address 127.0.0.1:39292)
0808 10:42:36.547150 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.547164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.547242 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.548104 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.548108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.548124 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:36.548204 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.548211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.549301 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.549303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.550611 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.550621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.551789 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.551795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.551944 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.551968 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.552039 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.552120 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"mutex_wait_us":24,"thread_start_us":96,"threads_started":1}
I20260808 10:42:36.552847 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.546332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.1.194.67:44835)
0808 10:42:36.547740 (+ 1408us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.547758 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.547775 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:42:36.547830 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.547833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.547952 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.548108 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:36.548324 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.549147 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.550735 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.551501 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.551593 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.551719 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.552569 (+ 850us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.552679 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1308,"thread_start_us":160,"threads_started":1}
I20260808 10:42:36.553553 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.545811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.1.194.65:37473)
0808 10:42:36.547073 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.547087 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.547109 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:42:36.547174 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.547177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.547310 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.547460 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:36.547788 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.548973 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.550572 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.551439 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.551567 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.551719 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.553271 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.553363 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1138,"thread_start_us":132,"threads_started":1}
I20260808 10:42:36.552168 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.545661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37473 (local address 127.0.0.1:51040)
0808 10:42:36.546553 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.546606 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.546740 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.547439 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.547442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.547461 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:36.547632 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.547639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.549343 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.549348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.550443 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.550452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.551776 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.551783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.551913 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.551932 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.551998 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.552055 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":31,"thread_start_us":97,"threads_started":1}
I20260808 10:42:36.555383 18696 tablet_service.cc:1467] Processing CreateTablet for tablet 3128f2b8b5844f459b5806626c27e834 (DEFAULT_TABLE table=test-table [id=1e798ad3199e4050915a8bee41162dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:36.555660 18838 tablet_service.cc:1467] Processing CreateTablet for tablet 3128f2b8b5844f459b5806626c27e834 (DEFAULT_TABLE table=test-table [id=1e798ad3199e4050915a8bee41162dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:36.556335 18696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128f2b8b5844f459b5806626c27e834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:36.556195 18556 tablet_service.cc:1467] Processing CreateTablet for tablet 3128f2b8b5844f459b5806626c27e834 (DEFAULT_TABLE table=test-table [id=1e798ad3199e4050915a8bee41162dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:36.556675 18838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128f2b8b5844f459b5806626c27e834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:36.557112 18556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128f2b8b5844f459b5806626c27e834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:36.562992 18925 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Bootstrap starting.
I20260808 10:42:36.563999 18927 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap starting.
I20260808 10:42:36.565549 18925 tablet_bootstrap.cc:654] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:36.565668 18927 tablet_bootstrap.cc:654] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:36.565851 18926 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Bootstrap starting.
I20260808 10:42:36.566393 18925 log.cc:824] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:36.566855 18927 log.cc:824] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:36.568512 18925 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: No bootstrap required, opened a new log
I20260808 10:42:36.568590 18926 tablet_bootstrap.cc:654] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:36.568725 18925 ts_tablet_manager.cc:1397] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:42:36.568789 18927 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: No bootstrap required, opened a new log
I20260808 10:42:36.568962 18927 ts_tablet_manager.cc:1397] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 10:42:36.569619 18926 log.cc:824] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:36.571480 18926 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: No bootstrap required, opened a new log
I20260808 10:42:36.571657 18926 ts_tablet_manager.cc:1397] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 10:42:36.572626 18925 raft_consensus.cc:348] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.572872 18925 raft_consensus.cc:374] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:36.572924 18925 raft_consensus.cc:729] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7191491dbf49c28e0470c66c345217, State: Initialized, Role: FOLLOWER
I20260808 10:42:36.572763 18927 raft_consensus.cc:348] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.572989 18927 raft_consensus.cc:374] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:36.573045 18927 raft_consensus.cc:729] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0534a64ea264cfbac23bea2636b8a24, State: Initialized, Role: FOLLOWER
I20260808 10:42:36.573421 18925 consensus_queue.cc:260] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.573448 18927 consensus_queue.cc:260] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.574330 18925 ts_tablet_manager.cc:1428] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:42:36.574394 18927 ts_tablet_manager.cc:1428] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:42:36.574774 18763 heartbeater.cc:503] Master 127.1.194.126:37735 was elected leader, sending a full tablet report...
I20260808 10:42:36.575258 18625 heartbeater.cc:503] Master 127.1.194.126:37735 was elected leader, sending a full tablet report...
I20260808 10:42:36.575369 18926 raft_consensus.cc:348] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.575605 18926 raft_consensus.cc:374] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:36.575659 18926 raft_consensus.cc:729] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4027456275494db3a5aa5323b1e40c09, State: Initialized, Role: FOLLOWER
I20260808 10:42:36.576349 18926 consensus_queue.cc:260] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.577020 18908 heartbeater.cc:503] Master 127.1.194.126:37735 was elected leader, sending a full tablet report...
I20260808 10:42:36.577376 18926 ts_tablet_manager.cc:1428] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 10:42:36.603670 18932 raft_consensus.cc:482] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:36.603873 18932 raft_consensus.cc:504] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.605005 18932 leader_election.cc:288] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473), 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835)
I20260808 10:42:36.609457 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.605271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44835 (local address 127.1.194.66:33425)
0808 10:42:36.605434 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.605452 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.605593 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.605949 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.605955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.605974 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:36.606107 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.606114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.607100 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.607104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.608040 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.608050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.608998 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.609002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.609112 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.609138 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.609224 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.609305 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 10:42:36.609592 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.605470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:33425 (local address 127.1.194.67:44835)
0808 10:42:36.605632 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.605637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.605657 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:42:36.605713 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.605716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.605777 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.605876 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:36.606238 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.606988 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.608190 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.608871 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.608976 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.609021 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.609256 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.609319 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:42:36.609982 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.605401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:50067 (local address 127.1.194.65:37473)
0808 10:42:36.605600 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.605604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.605890 (+ 286us) server_negotiation.cc:407] Connection header received
0808 10:42:36.606066 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.606071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.606125 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.606181 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:36.606549 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.607346 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.608725 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.609297 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.609410 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.609452 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.609752 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.609850 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:42:36.610385 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.605271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37473 (local address 127.1.194.66:50067)
0808 10:42:36.605792 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.605803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:36.605927 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:36.606309 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:36.606312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:36.606326 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:36.606421 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.606426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.607504 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.607508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.608557 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:36.608569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.609430 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.609434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:36.609573 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.609599 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:42:36.610179 (+ 580us) client_negotiation.cc:240] Negotiation successful
0808 10:42:36.610261 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":131,"thread_start_us":172,"threads_started":1}
I20260808 10:42:36.610327 18859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128f2b8b5844f459b5806626c27e834" candidate_uuid: "6d7191491dbf49c28e0470c66c345217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4027456275494db3a5aa5323b1e40c09" is_pre_election: true
I20260808 10:42:36.610661 18859 raft_consensus.cc:2454] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7191491dbf49c28e0470c66c345217 in term 0.
I20260808 10:42:36.611078 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128f2b8b5844f459b5806626c27e834" candidate_uuid: "6d7191491dbf49c28e0470c66c345217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0534a64ea264cfbac23bea2636b8a24" is_pre_election: true
I20260808 10:42:36.611245 18649 leader_election.cc:302] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4027456275494db3a5aa5323b1e40c09, 6d7191491dbf49c28e0470c66c345217; no voters:
I20260808 10:42:36.611339 18576 raft_consensus.cc:2454] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7191491dbf49c28e0470c66c345217 in term 0.
I20260808 10:42:36.611581 18932 raft_consensus.cc:2781] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:42:36.611697 18932 raft_consensus.cc:482] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:42:36.611763 18932 raft_consensus.cc:3037] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:36.612890 18932 raft_consensus.cc:504] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.613348 18932 leader_election.cc:288] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [CANDIDATE]: Term 1 election: Requested vote from peers e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473), 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835)
I20260808 10:42:36.613706 18859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128f2b8b5844f459b5806626c27e834" candidate_uuid: "6d7191491dbf49c28e0470c66c345217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4027456275494db3a5aa5323b1e40c09"
I20260808 10:42:36.613706 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128f2b8b5844f459b5806626c27e834" candidate_uuid: "6d7191491dbf49c28e0470c66c345217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0534a64ea264cfbac23bea2636b8a24"
I20260808 10:42:36.613839 18576 raft_consensus.cc:3037] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:36.613839 18859 raft_consensus.cc:3037] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:36.615234 18859 raft_consensus.cc:2454] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7191491dbf49c28e0470c66c345217 in term 1.
I20260808 10:42:36.615234 18576 raft_consensus.cc:2454] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7191491dbf49c28e0470c66c345217 in term 1.
I20260808 10:42:36.615568 18648 leader_election.cc:302] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7191491dbf49c28e0470c66c345217, e0534a64ea264cfbac23bea2636b8a24; no voters:
I20260808 10:42:36.615785 18932 raft_consensus.cc:2781] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:36.616024 18932 raft_consensus.cc:686] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7191491dbf49c28e0470c66c345217, State: Running, Role: LEADER
I20260808 10:42:36.616393 18932 consensus_queue.cc:237] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:36.619513 18417 catalog_manager.cc:5495] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d7191491dbf49c28e0470c66c345217 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "6d7191491dbf49c28e0470c66c345217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } health_report { overall_health: UNKNOWN } } }
I20260808 10:42:36.631474 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.627405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.1.194.66:34851)
0808 10:42:36.627554 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:36.627559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:36.627575 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:42:36.627802 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:36.627805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:36.627872 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:36.627989 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:36.628258 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.628808 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.629856 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:36.630117 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:36.630238 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:36.630567 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:36.631011 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:42:36.631161 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:36.631237 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 10:42:36.631297 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:42:36.637706 18859 raft_consensus.cc:1261] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Refusing update from remote peer 6d7191491dbf49c28e0470c66c345217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:36.637768 18576 raft_consensus.cc:1261] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Refusing update from remote peer 6d7191491dbf49c28e0470c66c345217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:36.638512 18935 consensus_queue.cc:1035] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:36.638731 18932 consensus_queue.cc:1035] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:36.649940 18940 mvcc.cc:205] Tried to move back new op lower bound from 7316216859180101632 to 7316216859103244288. Current Snapshot: MvccSnapshot[applied={T|T < 7316216859180101632}]
I20260808 10:42:36.655282 18417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:36.655442 18417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:36.657984 18417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Sending DeleteTablet for 3 replicas of tablet 3128f2b8b5844f459b5806626c27e834
I20260808 10:42:36.658885 18696 tablet_service.cc:1514] Processing DeleteTablet for tablet 3128f2b8b5844f459b5806626c27e834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:52042
I20260808 10:42:36.658905 18556 tablet_service.cc:1514] Processing DeleteTablet for tablet 3128f2b8b5844f459b5806626c27e834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:51040
I20260808 10:42:36.658959 18838 tablet_service.cc:1514] Processing DeleteTablet for tablet 3128f2b8b5844f459b5806626c27e834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:39292
I20260808 10:42:36.659446 18949 tablet_replica.cc:330] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: stopping tablet replica
I20260808 10:42:36.659684 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18489
I20260808 10:42:36.661365 18949 raft_consensus.cc:2229] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:42:36.661906 18949 raft_consensus.cc:2258] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:36.664278 18949 ts_tablet_manager.cc:1905] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:36.667963 18950 tablet_replica.cc:330] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: stopping tablet replica
I20260808 10:42:36.668505 18950 raft_consensus.cc:2229] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:36.668648 18949 ts_tablet_manager.cc:1918] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:36.668751 18949 log.cc:1196] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/3128f2b8b5844f459b5806626c27e834
I20260808 10:42:36.668902 18950 raft_consensus.cc:2258] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:36.669116 18949 ts_tablet_manager.cc:1939] T 3128f2b8b5844f459b5806626c27e834 P 6d7191491dbf49c28e0470c66c345217: Deleting consensus metadata
I20260808 10:42:36.670630 18407 catalog_manager.cc:4841] TS 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): tablet 3128f2b8b5844f459b5806626c27e834 (table test-table [id=1e798ad3199e4050915a8bee41162dbd]) successfully deleted
I20260808 10:42:36.674546 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18630
W20260808 10:42:36.674651 18408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:37473 (error 108)
I20260808 10:42:36.677300 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:36.676565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37473 (local address 127.0.0.1:51044)
0808 10:42:36.677109 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:36.677194 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426}
W20260808 10:42:36.677762 18408 catalog_manager.cc:4584] TS e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3128f2b8b5844f459b5806626c27e834: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37473: connect: Connection refused (error 111)
I20260808 10:42:36.678092 18950 ts_tablet_manager.cc:1905] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:36.680299 18950 ts_tablet_manager.cc:1918] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:36.680387 18950 log.cc:1196] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/3128f2b8b5844f459b5806626c27e834
I20260808 10:42:36.680733 18950 ts_tablet_manager.cc:1939] T 3128f2b8b5844f459b5806626c27e834 P 4027456275494db3a5aa5323b1e40c09: Deleting consensus metadata
I20260808 10:42:36.681999 18409 catalog_manager.cc:4841] TS 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): tablet 3128f2b8b5844f459b5806626c27e834 (table test-table [id=1e798ad3199e4050915a8bee41162dbd]) successfully deleted
I20260808 10:42:36.682861 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18768
I20260808 10:42:36.689670 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18388
I20260808 10:42:56.697561 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37735
--webserver_interface=127.1.194.126
--webserver_port=38473
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:37735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:42:56.815205 18953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:56.815512 18953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:56.815594 18953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:56.819672 18953 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:42:56.819754 18953 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:42:56.819783 18953 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:42:56.819818 18953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:42:56.819856 18953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:56.819916 18953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:56.819962 18953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:42:56.819993 18953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:42:56.825644 18953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:37735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=38473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:56.826722 18953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:56.827739 18953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:42:56.841809 18953 server_base.cc:1034] running on GCE node
W20260808 10:42:56.841735 18960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:56.841754 18962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:56.841730 18959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:56.842658 18953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:56.843899 18953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:56.845091 18953 hybrid_clock.cc:648] HybridClock initialized: now 1786185776845074 us; error 30 us; skew 500 ppm
Aug 08 10:42:56 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185776, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:56.854151 18953 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:42:56.855240 18953 webserver.cc:460] Webserver started at http://127.1.194.126:38473/ using document root <none> and password file <none>
I20260808 10:42:56.855875 18953 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:56.855984 18953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:56.860049 18953 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:42:56.862304 18970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:56.863592 18953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:56.863762 18953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "f084c21f2bf04dd98f1eac75595c9760"
format_stamp: "Formatted at 2026-08-08 10:42:35 on dist-test-slave-ss4n"
server_key: "835d74febcff69bf2b5de021063f661f"
server_key_iv: "43194476d2ea256a3eeb65598d543095"
server_key_version: "encryptionkey@0"
I20260808 10:42:56.864382 18953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:56.879237 18953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:56.882649 18953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:56.882856 18953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:56.890367 18953 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:37735
I20260808 10:42:56.890655 19023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:37735 every 8 connection(s)
I20260808 10:42:56.891556 18953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:42:56.896273 19024 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:42:56.896363 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 18953
I20260808 10:42:56.896917 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:37473
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:37735
--webserver_port=39305
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:56.898954 19024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap starting.
I20260808 10:42:56.917981 19024 log.cc:824] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:56.927877 19024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:56.928318 19024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap complete.
I20260808 10:42:56.933298 19024 raft_consensus.cc:348] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:56.933827 19024 raft_consensus.cc:729] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f084c21f2bf04dd98f1eac75595c9760, State: Initialized, Role: FOLLOWER
I20260808 10:42:56.934340 19024 consensus_queue.cc:260] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:56.934492 19024 raft_consensus.cc:388] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:42:56.934599 19024 raft_consensus.cc:482] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:42:56.934768 19024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:42:56.936844 19024 raft_consensus.cc:504] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:56.937215 19024 leader_election.cc:302] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f084c21f2bf04dd98f1eac75595c9760; no voters:
I20260808 10:42:56.937645 19024 leader_election.cc:288] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:42:56.937740 19029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:42:56.937943 19029 raft_consensus.cc:686] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 LEADER]: Becoming Leader. State: Replica: f084c21f2bf04dd98f1eac75595c9760, State: Running, Role: LEADER
I20260808 10:42:56.938274 19029 consensus_queue.cc:237] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:42:56.939025 19024 sys_catalog.cc:564] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:42:56.939710 19031 sys_catalog.cc:455] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f084c21f2bf04dd98f1eac75595c9760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } } }
I20260808 10:42:56.940153 19031 sys_catalog.cc:458] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:56.940039 19030 sys_catalog.cc:455] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f084c21f2bf04dd98f1eac75595c9760. Latest consensus state: current_term: 2 leader_uuid: "f084c21f2bf04dd98f1eac75595c9760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } } }
I20260808 10:42:56.940511 19030 sys_catalog.cc:458] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: This master's current role is: LEADER
I20260808 10:42:56.940943 19039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:42:56.946300 19039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:42:56.947043 19039 catalog_manager.cc:1259] Loaded cluster ID: a2bba6a0ea7545819590b809fccb26d6
I20260808 10:42:56.947139 19039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:42:56.949069 19039 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:42:56.949800 19039 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Loaded TSK: 0
I20260808 10:42:56.971374 19039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Generated new TSK 1
I20260808 10:42:56.971692 19039 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:42:57.023772 19026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:57.024019 19026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:57.024061 19026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:57.028234 19026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:57.028321 19026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:57.028410 19026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:42:57.033926 19026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:37473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=39305
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:57.034981 19026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:57.036018 19026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:57.046929 19052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:57.046888 19053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:57.046906 19055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:57.047508 19026 server_base.cc:1034] running on GCE node
I20260808 10:42:57.047953 19026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:57.048650 19026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:57.049839 19026 hybrid_clock.cc:648] HybridClock initialized: now 1786185777049809 us; error 43 us; skew 500 ppm
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:57.053056 19026 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:42:57.054256 19026 webserver.cc:460] Webserver started at http://127.1.194.65:39305/ using document root <none> and password file <none>
I20260808 10:42:57.054979 19026 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:57.055085 19026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:57.060122 19026 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:42:57.062815 19062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.064298 19026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:42:57.064430 19026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "e0534a64ea264cfbac23bea2636b8a24"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "dcd307ccd9a5e89d0691585270d3b5f9"
server_key_iv: "c7e8f33e8f9afe766ca662a14dab588f"
server_key_version: "encryptionkey@0"
I20260808 10:42:57.065132 19026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:57.085314 19026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:57.088513 19026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:57.088734 19026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:57.089363 19026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:57.090512 19069 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:42:57.095786 19026 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:42:57.095876 19026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 10:42:57.095968 19026 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:42:57.097817 19026 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:42:57.097889 19026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:57.098178 19069 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap starting.
I20260808 10:42:57.108721 19026 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:37473
I20260808 10:42:57.108996 19180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:37473 every 8 connection(s)
I20260808 10:42:57.109894 19026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:42:57.114941 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19026
I20260808 10:42:57.115413 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:34851
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:37735
--webserver_port=33149
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:42:57.119626 19069 log.cc:824] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:57.128961 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.111976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:38967)
0808 10:42:57.112539 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.112593 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.113606 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.115850 (+ 2244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.115862 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.116417 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:57.117114 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.117140 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.119136 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.119139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.120258 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.120274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.121230 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.121236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.121423 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.122150 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:57.122178 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:57.123638 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:57.125959 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.125965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.125977 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.126235 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.126589 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.126594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.126596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.126733 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.127294 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:57.127300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:57.127541 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.127723 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.128056 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":142,"threads_started":1}
I20260808 10:42:57.128948 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.111966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:38967 (local address 127.1.194.126:37735)
0808 10:42:57.113160 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.113179 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.113229 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:42:57.114028 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.114038 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.114329 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.114651 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:57.117368 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.118988 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.120419 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.121066 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.121271 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.123840 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:57.123845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:57.123856 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:57.123883 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:57.125835 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.126357 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.126365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.126374 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.126489 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.126830 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.126835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.126837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.127175 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:57.127318 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.127833 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.128091 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1031,"thread_start_us":113,"threads_started":1}
I20260808 10:42:57.129576 19069 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:42:57.130074 19069 tablet_bootstrap.cc:492] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap complete.
I20260808 10:42:57.131328 19181 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37735
I20260808 10:42:57.131556 19181 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:57.131560 19069 ts_tablet_manager.cc:1397] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Time spent bootstrapping tablet: real 0.034s user 0.017s sys 0.014s
I20260808 10:42:57.132118 19181 heartbeater.cc:511] Master 127.1.194.126:37735 requested a full tablet report, sending...
I20260808 10:42:57.134451 18987 ts_manager.cc:194] Registered new tserver with Master: e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473)
I20260808 10:42:57.134541 19069 raft_consensus.cc:348] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:57.135049 19069 raft_consensus.cc:729] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0534a64ea264cfbac23bea2636b8a24, State: Initialized, Role: FOLLOWER
I20260808 10:42:57.135504 19069 consensus_queue.cc:260] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } }
I20260808 10:42:57.136462 19069 ts_tablet_manager.cc:1428] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:42:57.138130 18987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:38967
I20260808 10:42:57.141005 19181 heartbeater.cc:503] Master 127.1.194.126:37735 was elected leader, sending a full tablet report...
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185776, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.65@KRBTEST.COM
I20260808 10:42:57.148545 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.137193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37473 (local address 127.0.0.1:45746)
0808 10:42:57.137681 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.137732 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.137862 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.138451 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.138460 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.138900 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:57.139024 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.139032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.140198 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.140202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.140881 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.140891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.142526 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.142530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.142646 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.143234 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:57.143246 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:57.144818 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:57.146749 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.146758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.146764 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.147059 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.147338 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.147341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.147343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.147437 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.147917 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:57.147922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:57.148046 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.148190 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.148375 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":133,"threads_started":1}
I20260808 10:42:57.148912 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.137270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.1.194.65:37473)
0808 10:42:57.137967 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.137978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.138000 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:42:57.138066 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.138070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.138268 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.138405 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:57.139149 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.140083 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.140999 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.141587 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.141711 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.144958 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:57.144967 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:57.144977 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:57.144997 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:57.146637 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.147173 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.147177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.147178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.147252 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.147588 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.147590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.147591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.147808 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:57.147903 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.148295 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.148443 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":152,"threads_started":1}
I20260808 10:42:57.149894 19114 tablet_service.cc:1514] Processing DeleteTablet for tablet 3128f2b8b5844f459b5806626c27e834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:45746
I20260808 10:42:57.150899 19190 tablet_replica.cc:330] stopping tablet replica
I20260808 10:42:57.151199 19190 raft_consensus.cc:2229] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:57.151443 19190 raft_consensus.cc:2258] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:57.152499 19190 ts_tablet_manager.cc:1905] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:57.156186 19190 ts_tablet_manager.cc:1918] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:57.156303 19190 log.cc:1196] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal/wals/3128f2b8b5844f459b5806626c27e834
I20260808 10:42:57.156581 19190 ts_tablet_manager.cc:1939] T 3128f2b8b5844f459b5806626c27e834 P e0534a64ea264cfbac23bea2636b8a24: Deleting consensus metadata
I20260808 10:42:57.158293 18973 catalog_manager.cc:4841] TS e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473): tablet 3128f2b8b5844f459b5806626c27e834 (table test-table [id=1e798ad3199e4050915a8bee41162dbd]) successfully deleted
W20260808 10:42:57.236667 19185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:57.236922 19185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:57.236965 19185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:57.241123 19185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:57.241184 19185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:57.241242 19185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:42:57.246572 19185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:34851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=33149
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:57.247534 19185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:57.248442 19185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:57.257933 19195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:57.258008 19185 server_base.cc:1034] running on GCE node
W20260808 10:42:57.257882 19198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:57.257889 19196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:57.258514 19185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:57.259083 19185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:57.260269 19185 hybrid_clock.cc:648] HybridClock initialized: now 1786185777260234 us; error 46 us; skew 500 ppm
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:57.263113 19185 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:42:57.263903 19185 webserver.cc:460] Webserver started at http://127.1.194.66:33149/ using document root <none> and password file <none>
I20260808 10:42:57.264480 19185 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:57.264581 19185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:57.268417 19185 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:42:57.270538 19205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.271817 19185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:42:57.271945 19185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "6d7191491dbf49c28e0470c66c345217"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "286e4af6ac5b58636e509c2e489c39d7"
server_key_iv: "2b0a315f1dafb755102e4a9dd36872f3"
server_key_version: "encryptionkey@0"
I20260808 10:42:57.272478 19185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:57.286760 19185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:57.290060 19185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:57.290259 19185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:57.290845 19185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:57.291713 19185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:57.291779 19185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.291847 19185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:57.291898 19185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.300803 19185 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:34851
I20260808 10:42:57.300837 19322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:34851 every 8 connection(s)
I20260808 10:42:57.301921 19185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:42:57.304903 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19185
I20260808 10:42:57.305505 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:44835
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:37735
--webserver_port=37963
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:57.318113 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.303805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:42379 (local address 127.1.194.126:37735)
0808 10:42:57.303949 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.303953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.304880 (+ 927us) server_negotiation.cc:407] Connection header received
0808 10:42:57.306863 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.306866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.306913 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.306973 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:57.308308 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.309147 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.310435 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.310867 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.311001 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.313848 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:57.313855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:57.313860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:57.313897 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:57.315705 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.316230 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.316235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.316239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.316341 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.316690 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.316695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.316698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.316998 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:57.317127 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.317692 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.317854 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:42:57.318782 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.304205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:42379)
0808 10:42:57.304685 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.304748 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.306613 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.307174 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.307185 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.307649 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:57.308132 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.308145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.309285 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.309287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.310304 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.310314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.311045 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.311049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.311187 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.311843 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:57.311858 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:57.313665 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:57.315837 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.315842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.315853 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.316111 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.316444 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.316446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.316448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.316577 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.317116 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:57.317121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:57.317355 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.317589 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.317850 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":119,"threads_started":1}
I20260808 10:42:57.320056 19323 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37735
I20260808 10:42:57.320353 19323 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:57.320924 19323 heartbeater.cc:511] Master 127.1.194.126:37735 requested a full tablet report, sending...
I20260808 10:42:57.322191 18987 ts_manager.cc:194] Registered new tserver with Master: 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
I20260808 10:42:57.322813 18987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:42379
W20260808 10:42:57.423689 19326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:42:57.423964 19326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:42:57.424007 19326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:42:57.428315 19326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:42:57.428385 19326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:42:57.428473 19326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:42:57.433864 19326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:44835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=37963
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:42:57.434958 19326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:42:57.435981 19326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:42:57.445914 19331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:57.445915 19334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:42:57.445915 19332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:42:57.447188 19326 server_base.cc:1034] running on GCE node
I20260808 10:42:57.447634 19326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:42:57.448263 19326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:42:57.449452 19326 hybrid_clock.cc:648] HybridClock initialized: now 1786185777449440 us; error 29 us; skew 500 ppm
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:42:57.452270 19326 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:42:57.453215 19326 webserver.cc:460] Webserver started at http://127.1.194.67:37963/ using document root <none> and password file <none>
I20260808 10:42:57.453864 19326 fs_manager.cc:359] Metadata directory not provided
I20260808 10:42:57.453981 19326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:42:57.457890 19326 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 10:42:57.459923 19342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.461141 19326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:42:57.461273 19326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "4027456275494db3a5aa5323b1e40c09"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "c51f59f8bf02fd8cec90f352fc0abbeb"
server_key_iv: "91b599424d360af1b1f2eacf3bb36df8"
server_key_version: "encryptionkey@0"
I20260808 10:42:57.461871 19326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:42:57.490669 19326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:42:57.493718 19326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:42:57.493925 19326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:42:57.494524 19326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:42:57.495364 19326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:42:57.495437 19326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.495508 19326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:42:57.495555 19326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:42:57.504570 19326 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:44835
I20260808 10:42:57.504662 19459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:44835 every 8 connection(s)
I20260808 10:42:57.505726 19326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:42:57.513777 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19326
Aug 08 10:42:57 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:42:57.519761 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.507525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:43865 (local address 127.1.194.126:37735)
0808 10:42:57.507698 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.507703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.508530 (+ 827us) server_negotiation.cc:407] Connection header received
0808 10:42:57.509808 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.509811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.509858 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.509934 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:57.511125 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.511654 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.512724 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.512932 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.513006 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.515920 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:57.515923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:57.515927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:57.515988 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:57.517637 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.518191 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.518195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.518197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.518290 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.518653 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.518657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.518658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.518844 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:57.518955 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.519474 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.519594 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:42:57.520628 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.507929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:43865)
0808 10:42:57.508373 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.508416 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.509585 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.510097 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.510104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.510453 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:42:57.510981 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.510994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.511764 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.511766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.512611 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.512619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.513058 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.513061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.513199 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.513928 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:42:57.513953 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:42:57.515743 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:42:57.517763 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.517769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.517786 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.518067 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.518385 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:42:57.518389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:42:57.518391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:42:57.518543 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:42:57.518955 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:42:57.518963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:42:57.519226 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.519432 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.519793 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260808 10:42:57.522069 19460 heartbeater.cc:347] Connected to a master server at 127.1.194.126:37735
I20260808 10:42:57.522360 19460 heartbeater.cc:464] Registering TS with master...
I20260808 10:42:57.522974 19460 heartbeater.cc:511] Master 127.1.194.126:37735 requested a full tablet report, sending...
I20260808 10:42:57.523531 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.514512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.1.194.126:37735)
0808 10:42:57.514817 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.514821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.514971 (+ 150us) server_negotiation.cc:407] Connection header received
0808 10:42:57.515053 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.515058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.515112 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.515209 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:57.515943 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.516759 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.517803 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.518036 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.518145 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.519558 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:57.519567 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:57.519570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:57.519592 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:57.521599 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.522123 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.522126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.522130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.522234 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.522544 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.522547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.522549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.522770 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:57.522984 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.523227 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.523362 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":76,"threads_started":1}
I20260808 10:42:57.524197 18986 ts_manager.cc:194] Registered new tserver with Master: 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835)
I20260808 10:42:57.524938 18986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:43865
I20260808 10:42:57.525396 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:42:57.529696 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.526160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.1.194.126:37735)
0808 10:42:57.526277 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.526281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.526498 (+ 217us) server_negotiation.cc:407] Connection header received
0808 10:42:57.526686 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.526689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.526786 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.526890 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:57.527125 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.527625 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.528504 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.528750 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.528855 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.529172 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:57.529288 (+ 116us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 10:42:57.529568 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49660: token expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 10:42:57.529762 19463 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49660: token expired
W20260808 10:42:57.530095 18913 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.194.126:37735)
I20260808 10:42:57.538173 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.530462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.1.194.126:37735)
0808 10:42:57.530609 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.530612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.530624 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:42:57.530683 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.530685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.530716 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.530777 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:42:57.531387 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.531914 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.532697 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.532895 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.532979 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.534329 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:42:57.534332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:42:57.534334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:42:57.534354 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:42:57.536183 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.536726 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.536729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.536731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.536823 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:42:57.537159 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:42:57.537164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:42:57.537168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:42:57.537394 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:42:57.537558 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.537884 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.538010 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:42:57.540940 18987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:42:57.544770 18987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:42:57.552937 19114 tablet_service.cc:1467] Processing CreateTablet for tablet 6cca9fb0060a452da7694594957c7826 (DEFAULT_TABLE table=test-table [id=a9fbeb374ccb4686802cb05664bc0916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:57.553647 19114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cca9fb0060a452da7694594957c7826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:57.556285 19069 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap starting.
I20260808 10:42:57.557195 19069 tablet_bootstrap.cc:654] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:57.558308 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.552256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:34851 (local address 127.0.0.1:57334)
0808 10:42:57.552580 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.552594 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.552678 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.553631 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.553655 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.553676 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:57.553771 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.553777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.555144 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.555147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.556584 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.556596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.557856 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.557860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.557958 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.557998 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.558090 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.558165 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":117,"thread_start_us":122,"threads_started":1}
I20260808 10:42:57.558562 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.552437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.1.194.66:34851)
0808 10:42:57.552885 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.552896 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.552914 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:42:57.552996 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.553002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.553228 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.553402 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:57.553910 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.555019 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.556714 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.557458 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.557602 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.557770 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.558127 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.558221 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":158,"threads_started":1}
I20260808 10:42:57.558593 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.551801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44835 (local address 127.0.0.1:59362)
0808 10:42:57.551958 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.551973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.552053 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.553399 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.553402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.553420 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:57.553571 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.553579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.555028 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.555032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.556463 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.556474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.558167 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.558171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.558286 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.558311 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.558372 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.558463 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":40}
I20260808 10:42:57.558833 19069 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: No bootstrap required, opened a new log
I20260808 10:42:57.558933 19069 ts_tablet_manager.cc:1397] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:42:57.559239 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.552032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.1.194.67:44835)
0808 10:42:57.552873 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.552888 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.552914 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:42:57.552996 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.553002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.553228 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.553402 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:57.553703 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.554925 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.556608 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.558004 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.558777 (+ 773us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.558922 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.558978 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.559077 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":186,"threads_started":1}
I20260808 10:42:57.559405 19069 raft_consensus.cc:348] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.559533 19069 raft_consensus.cc:374] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:57.559581 19069 raft_consensus.cc:729] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0534a64ea264cfbac23bea2636b8a24, State: Initialized, Role: FOLLOWER
I20260808 10:42:57.559723 19069 consensus_queue.cc:260] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.559971 19069 ts_tablet_manager.cc:1428] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:42:57.562258 19256 tablet_service.cc:1467] Processing CreateTablet for tablet 6cca9fb0060a452da7694594957c7826 (DEFAULT_TABLE table=test-table [id=a9fbeb374ccb4686802cb05664bc0916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:57.562537 19393 tablet_service.cc:1467] Processing CreateTablet for tablet 6cca9fb0060a452da7694594957c7826 (DEFAULT_TABLE table=test-table [id=a9fbeb374ccb4686802cb05664bc0916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:42:57.563283 19256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cca9fb0060a452da7694594957c7826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:57.563556 19393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cca9fb0060a452da7694594957c7826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:42:57.570097 19474 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Bootstrap starting.
I20260808 10:42:57.570464 19473 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Bootstrap starting.
I20260808 10:42:57.572762 19473 tablet_bootstrap.cc:654] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:57.572764 19474 tablet_bootstrap.cc:654] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Neither blocks nor log segments found. Creating new log.
I20260808 10:42:57.573586 19474 log.cc:824] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:57.573577 19473 log.cc:824] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Log is configured to *not* fsync() on all Append() calls
I20260808 10:42:57.575691 19474 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: No bootstrap required, opened a new log
I20260808 10:42:57.575734 19473 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: No bootstrap required, opened a new log
I20260808 10:42:57.575953 19473 ts_tablet_manager.cc:1397] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:42:57.575958 19474 ts_tablet_manager.cc:1397] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:42:57.579087 19473 raft_consensus.cc:348] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.579097 19474 raft_consensus.cc:348] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.579366 19473 raft_consensus.cc:374] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:57.579367 19474 raft_consensus.cc:374] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:42:57.579423 19473 raft_consensus.cc:729] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4027456275494db3a5aa5323b1e40c09, State: Initialized, Role: FOLLOWER
I20260808 10:42:57.579427 19474 raft_consensus.cc:729] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7191491dbf49c28e0470c66c345217, State: Initialized, Role: FOLLOWER
I20260808 10:42:57.579869 19474 consensus_queue.cc:260] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.579869 19473 consensus_queue.cc:260] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.580600 19460 heartbeater.cc:503] Master 127.1.194.126:37735 was elected leader, sending a full tablet report...
I20260808 10:42:57.580770 19323 heartbeater.cc:503] Master 127.1.194.126:37735 was elected leader, sending a full tablet report...
I20260808 10:42:57.580850 19473 ts_tablet_manager.cc:1428] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:42:57.580870 19474 ts_tablet_manager.cc:1428] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 10:42:57.611317 19182 tablet.cc:2369] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:42:57.686610 19187 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:42:57.686867 19187 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.688054 19187 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
I20260808 10:42:57.692150 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.688546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:53321 (local address 127.1.194.66:34851)
0808 10:42:57.688676 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.688687 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.688807 (+ 120us) server_negotiation.cc:407] Connection header received
0808 10:42:57.688948 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.688954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.689013 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.689073 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:57.689498 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.689995 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.691067 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.691505 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.691639 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.691690 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.691943 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.692011 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:42:57.692169 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.688296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44835 (local address 127.1.194.65:48219)
0808 10:42:57.688650 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.688665 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.688819 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.689157 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.689160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.689191 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:57.689333 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.689338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.690150 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.690153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.690975 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.691010 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.691659 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.691663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.691801 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.691836 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.691934 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.692016 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":74,"thread_start_us":115,"threads_started":1}
I20260808 10:42:57.692155 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.688407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:48219 (local address 127.1.194.67:44835)
0808 10:42:57.688542 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.688550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.688771 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:42:57.688988 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.688991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.689029 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.689123 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:42:57.689451 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.690007 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.691135 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.691514 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.691657 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.691708 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.691949 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.692020 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:42:57.692217 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.688297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:34851 (local address 127.1.194.65:53321)
0808 10:42:57.688690 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.688702 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:42:57.688837 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:42:57.689111 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:42:57.689115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:42:57.689132 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:42:57.689334 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.689341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.690106 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.690109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.690926 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:42:57.690935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.691633 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.691636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:42:57.691790 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.691832 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:42:57.691927 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:42:57.692020 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":127,"spinlock_wait_cycles":21248,"thread_start_us":69,"threads_started":1}
I20260808 10:42:57.692777 19413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cca9fb0060a452da7694594957c7826" candidate_uuid: "e0534a64ea264cfbac23bea2636b8a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4027456275494db3a5aa5323b1e40c09" is_pre_election: true
I20260808 10:42:57.692785 19277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cca9fb0060a452da7694594957c7826" candidate_uuid: "e0534a64ea264cfbac23bea2636b8a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7191491dbf49c28e0470c66c345217" is_pre_election: true
I20260808 10:42:57.693069 19277 raft_consensus.cc:2454] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0534a64ea264cfbac23bea2636b8a24 in term 0.
I20260808 10:42:57.693080 19413 raft_consensus.cc:2454] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0534a64ea264cfbac23bea2636b8a24 in term 0.
I20260808 10:42:57.693589 19066 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4027456275494db3a5aa5323b1e40c09, e0534a64ea264cfbac23bea2636b8a24; no voters:
I20260808 10:42:57.693846 19187 raft_consensus.cc:2781] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:42:57.693943 19187 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:42:57.694005 19187 raft_consensus.cc:3037] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:57.695060 19187 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.695425 19187 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 1 election: Requested vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
I20260808 10:42:57.695906 19277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cca9fb0060a452da7694594957c7826" candidate_uuid: "e0534a64ea264cfbac23bea2636b8a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7191491dbf49c28e0470c66c345217"
I20260808 10:42:57.696045 19277 raft_consensus.cc:3037] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:57.696195 19413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cca9fb0060a452da7694594957c7826" candidate_uuid: "e0534a64ea264cfbac23bea2636b8a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4027456275494db3a5aa5323b1e40c09"
I20260808 10:42:57.696324 19413 raft_consensus.cc:3037] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:42:57.697582 19277 raft_consensus.cc:2454] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0534a64ea264cfbac23bea2636b8a24 in term 1.
I20260808 10:42:57.697698 19413 raft_consensus.cc:2454] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0534a64ea264cfbac23bea2636b8a24 in term 1.
I20260808 10:42:57.697960 19064 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7191491dbf49c28e0470c66c345217, e0534a64ea264cfbac23bea2636b8a24; no voters:
I20260808 10:42:57.698127 19187 raft_consensus.cc:2781] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:42:57.698348 19187 raft_consensus.cc:686] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 LEADER]: Becoming Leader. State: Replica: e0534a64ea264cfbac23bea2636b8a24, State: Running, Role: LEADER
I20260808 10:42:57.698681 19187 consensus_queue.cc:237] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:42:57.701794 18987 catalog_manager.cc:5495] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65). New cstate: current_term: 1 leader_uuid: "e0534a64ea264cfbac23bea2636b8a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } health_report { overall_health: HEALTHY } } }
I20260808 10:42:57.720901 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.717081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.1.194.65:37473)
0808 10:42:57.717388 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 10:42:57.717396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:42:57.717407 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:42:57.717526 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:42:57.717529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:42:57.717599 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:42:57.717731 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:42:57.717982 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.718502 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.719340 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:42:57.719570 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:42:57.719681 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:42:57.719953 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:42:57.720486 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:42:57.720605 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:42:57.720676 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:42:57.720729 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260808 10:42:57.724984 19277 raft_consensus.cc:1261] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Refusing update from remote peer e0534a64ea264cfbac23bea2636b8a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:57.725071 19413 raft_consensus.cc:1261] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Refusing update from remote peer e0534a64ea264cfbac23bea2636b8a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:42:57.725831 19481 consensus_queue.cc:1035] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:57.726048 19187 consensus_queue.cc:1035] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:42:57.734127 19486 mvcc.cc:205] Tried to move back new op lower bound from 7316216945554677760 to 7316216945456644096. Current Snapshot: MvccSnapshot[applied={T|T < 7316216945554677760}]
I20260808 10:42:57.734823 19485 mvcc.cc:205] Tried to move back new op lower bound from 7316216945554677760 to 7316216945456644096. Current Snapshot: MvccSnapshot[applied={T|T < 7316216945554677760}]
I20260808 10:42:57.741074 19487 mvcc.cc:205] Tried to move back new op lower bound from 7316216945554677760 to 7316216945456644096. Current Snapshot: MvccSnapshot[applied={T|T < 7316216945554677760}]
I20260808 10:42:57.743009 18985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:57.743207 18985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:42:57.745698 18985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Sending DeleteTablet for 3 replicas of tablet 6cca9fb0060a452da7694594957c7826
I20260808 10:42:57.746393 19393 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cca9fb0060a452da7694594957c7826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:57 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:59362
I20260808 10:42:57.746503 19256 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cca9fb0060a452da7694594957c7826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:57 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:57334
I20260808 10:42:57.746821 19114 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cca9fb0060a452da7694594957c7826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:42:57 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:45746
I20260808 10:42:57.746855 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 19026
I20260808 10:42:57.748230 19494 tablet_replica.cc:330] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: stopping tablet replica
I20260808 10:42:57.748634 19494 raft_consensus.cc:2229] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:57.748996 19494 raft_consensus.cc:2258] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:57.749522 19495 tablet_replica.cc:330] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: stopping tablet replica
I20260808 10:42:57.749876 19495 raft_consensus.cc:2229] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:42:57.750156 19495 raft_consensus.cc:2258] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:42:57.750720 19494 ts_tablet_manager.cc:1905] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:57.751401 19495 ts_tablet_manager.cc:1905] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:42:57.753718 19494 ts_tablet_manager.cc:1918] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:57.753827 19494 log.cc:1196] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/6cca9fb0060a452da7694594957c7826
I20260808 10:42:57.754222 19494 ts_tablet_manager.cc:1939] T 6cca9fb0060a452da7694594957c7826 P 4027456275494db3a5aa5323b1e40c09: Deleting consensus metadata
I20260808 10:42:57.754209 19495 ts_tablet_manager.cc:1918] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:42:57.754299 19495 log.cc:1196] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/6cca9fb0060a452da7694594957c7826
I20260808 10:42:57.754648 19495 ts_tablet_manager.cc:1939] T 6cca9fb0060a452da7694594957c7826 P 6d7191491dbf49c28e0470c66c345217: Deleting consensus metadata
I20260808 10:42:57.755654 18972 catalog_manager.cc:4841] TS 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): tablet 6cca9fb0060a452da7694594957c7826 (table test-table [id=a9fbeb374ccb4686802cb05664bc0916]) successfully deleted
I20260808 10:42:57.755671 18974 catalog_manager.cc:4841] TS 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): tablet 6cca9fb0060a452da7694594957c7826 (table test-table [id=a9fbeb374ccb4686802cb05664bc0916]) successfully deleted
W20260808 10:42:57.756165 18973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:37473 (error 108)
I20260808 10:42:57.756585 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 19185
I20260808 10:42:57.758296 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:42:57.757579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37473 (local address 127.0.0.1:45760)
0808 10:42:57.758064 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 10:42:57.758183 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378}
W20260808 10:42:57.758517 18973 catalog_manager.cc:4584] TS e0534a64ea264cfbac23bea2636b8a24 (127.1.194.65:37473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6cca9fb0060a452da7694594957c7826: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37473: connect: Connection refused (error 111)
I20260808 10:42:57.763707 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 19326
I20260808 10:42:57.772244 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 18953
I20260808 10:43:17.780035 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37735
--webserver_interface=127.1.194.126
--webserver_port=38473
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:37735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:17.893386 19497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:17.893757 19497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:17.893848 19497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:17.897853 19497 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:43:17.897940 19497 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:43:17.897965 19497 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:43:17.897986 19497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:17.898031 19497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:17.898067 19497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:17.898097 19497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:17.898132 19497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:17.903371 19497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:37735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:37735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=38473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:17.904515 19497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:17.905586 19497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:17.914685 19503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:17.914744 19506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:17.914934 19497 server_base.cc:1034] running on GCE node
W20260808 10:43:17.914988 19504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:17.915822 19497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:17.917337 19497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:17.918512 19497 hybrid_clock.cc:648] HybridClock initialized: now 1786185797918490 us; error 38 us; skew 500 ppm
Aug 08 10:43:17 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185797, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:17.922086 19497 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:43:17.923005 19497 webserver.cc:460] Webserver started at http://127.1.194.126:38473/ using document root <none> and password file <none>
I20260808 10:43:17.923580 19497 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:17.923647 19497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:17.927492 19497 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:43:17.929571 19513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:17.930733 19497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:17.930866 19497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "f084c21f2bf04dd98f1eac75595c9760"
format_stamp: "Formatted at 2026-08-08 10:42:35 on dist-test-slave-ss4n"
server_key: "835d74febcff69bf2b5de021063f661f"
server_key_iv: "43194476d2ea256a3eeb65598d543095"
server_key_version: "encryptionkey@0"
I20260808 10:43:17.931399 19497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:17.952440 19497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:17.955355 19497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:17.955564 19497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:17.962597 19497 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:37735
I20260808 10:43:17.962885 19568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:37735 every 8 connection(s)
I20260808 10:43:17.963680 19497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:17.967842 19569 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:43:17.968081 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19497
I20260808 10:43:17.968532 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:37473
--local_ip_for_outbound_sockets=127.1.194.65
--tserver_master_addrs=127.1.194.126:37735
--webserver_port=39305
--webserver_interface=127.1.194.65
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:43:17.969625 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap starting.
I20260808 10:43:17.994549 19569 log.cc:824] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:18.008745 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:43:18.009222 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Bootstrap complete.
I20260808 10:43:18.014878 19569 raft_consensus.cc:348] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:43:18.015405 19569 raft_consensus.cc:729] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f084c21f2bf04dd98f1eac75595c9760, State: Initialized, Role: FOLLOWER
I20260808 10:43:18.015936 19569 consensus_queue.cc:260] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:43:18.016098 19569 raft_consensus.cc:388] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:18.016170 19569 raft_consensus.cc:482] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:18.016273 19569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:43:18.018772 19569 raft_consensus.cc:504] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:43:18.019150 19569 leader_election.cc:302] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f084c21f2bf04dd98f1eac75595c9760; no voters:
I20260808 10:43:18.019490 19569 leader_election.cc:288] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:43:18.019658 19573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:43:18.019948 19573 raft_consensus.cc:686] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [term 3 LEADER]: Becoming Leader. State: Replica: f084c21f2bf04dd98f1eac75595c9760, State: Running, Role: LEADER
I20260808 10:43:18.020371 19573 consensus_queue.cc:237] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } }
I20260808 10:43:18.020730 19569 sys_catalog.cc:564] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:18.021890 19576 sys_catalog.cc:455] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f084c21f2bf04dd98f1eac75595c9760. Latest consensus state: current_term: 3 leader_uuid: "f084c21f2bf04dd98f1eac75595c9760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } } }
I20260808 10:43:18.022007 19576 sys_catalog.cc:458] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:18.022687 19574 sys_catalog.cc:455] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f084c21f2bf04dd98f1eac75595c9760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f084c21f2bf04dd98f1eac75595c9760" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 37735 } } }
I20260808 10:43:18.022768 19574 sys_catalog.cc:458] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:18.022775 19583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:18.027220 19583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:18.027760 19583 catalog_manager.cc:1259] Loaded cluster ID: a2bba6a0ea7545819590b809fccb26d6
I20260808 10:43:18.027832 19583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:18.029623 19583 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:18.030207 19583 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Loaded TSK: 0
I20260808 10:43:18.030292 19583 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f084c21f2bf04dd98f1eac75595c9760: Loaded TSK: 1
F20260808 10:43:18.030881 19583 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0e708209ad google::LogMessage::Fail() at ??:0
@ 0x7f0e7082388a google::LogMessage::SendToLog() at ??:0
@ 0x7f0e708204de google::LogMessage::Flush() at ??:0
@ 0x7f0e70821a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0e734e418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0e734dcf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0e7b1897bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0e7b16399c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0e7b164178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f0e7b1a5df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f0e7b1c482b std::function<>::operator()() at ??:0
@ 0x7f0e7b163b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f0e7b164f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0e7b161579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f0e7b1a5640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0e72a723d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0e72a72c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f0e72a743f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0e72a63512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0e740a76db start_thread at ??:0
@ 0x7f0e6f31e61f clone at ??:0
W20260808 10:43:18.118893 19571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:18.119194 19571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:18.119239 19571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:18.123696 19571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:18.123740 19571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:18.123801 19571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:18.129379 19571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:37473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=39305
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:18.130429 19571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:18.131533 19571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:18.141323 19621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:18.141391 19622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:18.141367 19625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:18.141741 19571 server_base.cc:1034] running on GCE node
I20260808 10:43:18.142148 19571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:18.142807 19571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:18.144011 19571 hybrid_clock.cc:648] HybridClock initialized: now 1786185798143983 us; error 47 us; skew 500 ppm
Aug 08 10:43:18 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185798, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:18.146931 19571 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:43:18.147855 19571 webserver.cc:460] Webserver started at http://127.1.194.65:39305/ using document root <none> and password file <none>
I20260808 10:43:18.148409 19571 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:18.148496 19571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:18.152396 19571 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:18.154646 19632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.155933 19571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:18.156023 19571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "e0534a64ea264cfbac23bea2636b8a24"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "dcd307ccd9a5e89d0691585270d3b5f9"
server_key_iv: "c7e8f33e8f9afe766ca662a14dab588f"
server_key_version: "encryptionkey@0"
I20260808 10:43:18.156569 19571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:18.174260 19571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:18.178090 19571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:18.178293 19571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:18.178897 19571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:18.180015 19640 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:43:18.185413 19571 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:43:18.185509 19571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 10:43:18.185616 19571 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:43:18.187233 19571 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:43:18.187299 19571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:43:18.187583 19640 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap starting.
I20260808 10:43:18.198015 19571 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:37473
I20260808 10:43:18.198060 19753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:37473 every 8 connection(s)
I20260808 10:43:18.198966 19571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:43:18.207077 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19571
I20260808 10:43:18.207558 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:34851
--local_ip_for_outbound_sockets=127.1.194.66
--tserver_master_addrs=127.1.194.126:37735
--webserver_port=33149
--webserver_interface=127.1.194.66
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:43:18.209667 19640 log.cc:824] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:18.216802 19640 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:43:18.217216 19640 tablet_bootstrap.cc:492] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Bootstrap complete.
I20260808 10:43:18.218381 19640 ts_tablet_manager.cc:1397] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.010s
I20260808 10:43:18.220964 19640 raft_consensus.cc:348] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:18.221465 19640 raft_consensus.cc:729] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0534a64ea264cfbac23bea2636b8a24, State: Initialized, Role: FOLLOWER
I20260808 10:43:18.222011 19640 consensus_queue.cc:260] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:18.223018 19640 ts_tablet_manager.cc:1428] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
W20260808 10:43:18.294554 19756 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:43:18.200844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:60319)
0808 10:43:18.201312 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.201365 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:18.202208 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:18.293757 (+ 91549us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":138,"threads_started":1}
W20260808 10:43:18.295084 19754 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:37735 (0 consecutive failures): Network error: Failed to ping master at 127.1.194.126:37735: Client connection negotiation failed: client connection to 127.1.194.126:37735: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:43:18.295867 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.295610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:50607)
0808 10:43:18.295744 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.295809 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 10:43:18.296485 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.296288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:36845)
0808 10:43:18.296388 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.296439 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:43:18.296645 19754 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:43:18.344933 19757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:18.345317 19757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:18.345412 19757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:18.349963 19757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:18.350088 19757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:18.350193 19757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:18.356174 19757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:34851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=33149
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:18.357348 19757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:18.358438 19757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:18.369547 19767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:18.369557 19764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:18.369608 19757 server_base.cc:1034] running on GCE node
W20260808 10:43:18.369710 19765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:18.370404 19757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:18.371101 19757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:18.372303 19757 hybrid_clock.cc:648] HybridClock initialized: now 1786185798372286 us; error 30 us; skew 500 ppm
Aug 08 10:43:18 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185798, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:18.375725 19757 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:43:18.376700 19757 webserver.cc:460] Webserver started at http://127.1.194.66:33149/ using document root <none> and password file <none>
I20260808 10:43:18.377322 19757 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:18.377449 19757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:18.381800 19757 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:43:18.384083 19774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.385310 19757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:18.385543 19757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "6d7191491dbf49c28e0470c66c345217"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "286e4af6ac5b58636e509c2e489c39d7"
server_key_iv: "2b0a315f1dafb755102e4a9dd36872f3"
server_key_version: "encryptionkey@0"
I20260808 10:43:18.386158 19757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:18.402830 19757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:18.405886 19757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:18.406100 19757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:18.406704 19757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:18.407531 19757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:18.407603 19757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.407677 19757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:18.407720 19757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.416863 19757 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:34851
I20260808 10:43:18.416887 19889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:34851 every 8 connection(s)
I20260808 10:43:18.417893 19757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:18.421802 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.420027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:50857)
0808 10:43:18.420528 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.421015 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":148,"threads_started":1}
W20260808 10:43:18.422220 19890 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:37735 (0 consecutive failures): Network error: Failed to ping master at 127.1.194.126:37735: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
I20260808 10:43:18.422885 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.422615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:58015)
0808 10:43:18.422764 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.422823 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 10:43:18.423667 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.423380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:55705)
0808 10:43:18.423561 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.423614 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260808 10:43:18.424019 19890 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:43:18.426592 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19757
I20260808 10:43:18.427053 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:44835
--local_ip_for_outbound_sockets=127.1.194.67
--tserver_master_addrs=127.1.194.126:37735
--webserver_port=37963
--webserver_interface=127.1.194.67
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:18.542650 19893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:18.542927 19893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:18.542971 19893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:18.547057 19893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:18.547134 19893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:18.547194 19893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:18.552618 19893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:44835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=37963
--tserver_master_addrs=127.1.194.126:37735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:18.553824 19893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:18.554800 19893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:18.564491 19901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:18.564621 19898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:18.564734 19893 server_base.cc:1034] running on GCE node
W20260808 10:43:18.564491 19899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:18.565330 19893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:18.565984 19893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:18.567190 19893 hybrid_clock.cc:648] HybridClock initialized: now 1786185798567156 us; error 46 us; skew 500 ppm
Aug 08 10:43:18 dist-test-slave-ss4n krb5kdc[18375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185798, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:18.570050 19893 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:43:18.570864 19893 webserver.cc:460] Webserver started at http://127.1.194.67:37963/ using document root <none> and password file <none>
I20260808 10:43:18.571380 19893 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:18.571439 19893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:18.575218 19893 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:18.577387 19909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.578888 19893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:43:18.579001 19893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "4027456275494db3a5aa5323b1e40c09"
format_stamp: "Formatted at 2026-08-08 10:42:36 on dist-test-slave-ss4n"
server_key: "c51f59f8bf02fd8cec90f352fc0abbeb"
server_key_iv: "91b599424d360af1b1f2eacf3bb36df8"
server_key_version: "encryptionkey@0"
I20260808 10:43:18.579674 19893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:18.598271 19893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:18.601063 19893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:18.601212 19893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:18.601802 19893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:18.602622 19893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:18.602669 19893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.602711 19893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:18.602733 19893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:18.611573 19893 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:44835
I20260808 10:43:18.611605 20027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:44835 every 8 connection(s)
I20260808 10:43:18.612558 19893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:18.614496 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 19893
I20260808 10:43:18.617810 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.615189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:58903)
0808 10:43:18.615789 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.615924 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":164,"threads_started":1}
W20260808 10:43:18.618520 20028 heartbeater.cc:644] Failed to heartbeat to 127.1.194.126:37735 (0 consecutive failures): Network error: Failed to ping master at 127.1.194.126:37735: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
I20260808 10:43:18.619181 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.618964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:55995)
0808 10:43:18.619063 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.619121 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 10:43:18.619809 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:18.619593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:48105)
0808 10:43:18.619708 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:18.619753 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:43:18.620013 20028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:43:19.297855 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.297373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:47599)
0808 10:43:19.297678 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:19.297742 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":142,"threads_started":1}
I20260808 10:43:19.425019 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.424608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:49897)
0808 10:43:19.424853 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:19.424912 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":73,"threads_started":1}
I20260808 10:43:19.540448 20034 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:19.540767 20034 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:19.542080 20034 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
Aug 08 10:43:19 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185798, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.67@KRBTEST.COM
Aug 08 10:43:19 dist-test-slave-ss4n krb5kdc[18375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185798, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.66@KRBTEST.COM
I20260808 10:43:19.566002 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.542423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:34119 (local address 127.1.194.67:44835)
0808 10:43:19.542864 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:19.542879 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:19.542926 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:43:19.543840 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:19.543857 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:19.544305 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:19.544635 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:19.545813 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.546789 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.548064 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.548580 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.548775 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:19.560200 (+ 11425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:19.560210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:19.560229 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:19.560272 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:19.562910 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:19.563711 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:19.563721 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:19.563725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:19.563846 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:19.564313 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:19.564316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:19.564317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:19.564744 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:19.564895 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:19.565471 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 10:43:19.565782 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":69,"threads_started":1}
I20260808 10:43:19.566021 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.542563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:39171 (local address 127.1.194.66:34851)
0808 10:43:19.542918 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:19.542941 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:19.542971 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:43:19.543922 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:19.543929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:19.544305 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:19.544635 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:19.545767 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.546754 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.547952 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.548455 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.548652 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:19.560322 (+ 11670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:19.560330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:19.560345 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:19.560388 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:19.562910 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:19.563688 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:19.563696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:19.563709 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:19.563848 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:19.564253 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:19.564259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:19.564264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:19.564746 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:19.564900 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:19.565515 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 10:43:19.565805 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":152,"threads_started":1}
I20260808 10:43:19.566040 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.542257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44835 (local address 127.1.194.65:34119)
0808 10:43:19.542442 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:19.542457 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:19.542602 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:19.544707 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:19.544712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:19.545112 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:19.545589 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:19.545606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.546943 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.546948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:19.547867 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:19.547879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.548727 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.548731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:19.548823 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:19.557798 (+ 8975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:19.557821 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:19.559859 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:19.563092 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:19.563097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:19.563109 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:19.563553 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:19.563967 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:19.563973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:19.563979 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:19.564158 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:19.564875 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:19.564885 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:19.565148 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 10:43:19.565350 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:43:19.565813 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":151,"spinlock_wait_cycles":38784}
I20260808 10:43:19.566807 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.542336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:34851 (local address 127.1.194.65:39171)
0808 10:43:19.542734 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:19.542751 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:19.542849 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:19.545128 (+ 2279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:19.545131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:19.545303 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:19.545589 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:19.545606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.546935 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.546937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:19.547801 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:19.547811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:19.548596 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:19.548599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:19.548782 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:19.557798 (+ 9016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:19.557836 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:19.560109 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:19.563085 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:19.563092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:19.563109 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:19.563545 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:19.563980 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:19.563983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:19.563985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:19.564145 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:19.564873 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:19.564880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:19.565147 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 10:43:19.565350 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 10:43:19.565904 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":343,"spinlock_wait_cycles":33920,"thread_start_us":99,"threads_started":1}
W20260808 10:43:19.568740 19637 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
W20260808 10:43:19.568913 19635 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
I20260808 10:43:19.569018 19635 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0534a64ea264cfbac23bea2636b8a24; no voters: 4027456275494db3a5aa5323b1e40c09, 6d7191491dbf49c28e0470c66c345217
I20260808 10:43:19.569337 20034 raft_consensus.cc:2726] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:43:19.621294 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:19.620769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:56383)
0808 10:43:19.621060 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:19.621133 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
I20260808 10:43:20.299333 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:20.298743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:53141)
0808 10:43:20.299109 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:20.299189 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":94,"threads_started":1}
I20260808 10:43:20.426366 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:20.425914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:36985)
0808 10:43:20.426205 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:20.426261 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":91,"threads_started":1}
I20260808 10:43:20.622738 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:20.622123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:50071)
0808 10:43:20.622464 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:20.622536 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260808 10:43:21.300671 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:21.300246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:37355)
0808 10:43:21.300512 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:21.300571 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260808 10:43:21.384585 20043 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:21.384727 20043 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:21.385119 20043 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
W20260808 10:43:21.386128 19635 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
W20260808 10:43:21.386297 19637 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
I20260808 10:43:21.386391 19637 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0534a64ea264cfbac23bea2636b8a24; no voters: 4027456275494db3a5aa5323b1e40c09, 6d7191491dbf49c28e0470c66c345217
I20260808 10:43:21.386529 20043 raft_consensus.cc:2726] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:43:21.427513 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:21.427111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:55511)
0808 10:43:21.427370 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:21.427429 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
I20260808 10:43:21.623900 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:21.623503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:42047)
0808 10:43:21.623751 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:21.623808 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":87,"threads_started":1}
I20260808 10:43:22.302243 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:22.301785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:48315)
0808 10:43:22.302066 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:22.302130 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
I20260808 10:43:22.428669 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:22.428212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:55383)
0808 10:43:22.428490 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:22.428546 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":98,"threads_started":1}
I20260808 10:43:22.625161 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:22.624707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:37403)
0808 10:43:22.625000 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:22.625061 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
I20260808 10:43:23.303828 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:23.303281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:44051)
0808 10:43:23.303653 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:23.303724 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":103,"threads_started":1}
I20260808 10:43:23.430001 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:23.429549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:44817)
0808 10:43:23.429829 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:23.429892 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260808 10:43:23.626632 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:23.626096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:60575)
0808 10:43:23.626383 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:23.626525 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":78,"threads_started":1}
I20260808 10:43:24.305142 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:24.304716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:53693)
0808 10:43:24.304983 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:24.305042 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260808 10:43:24.431265 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:24.430813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:34997)
0808 10:43:24.431094 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:24.431169 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260808 10:43:24.606784 20055 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:24.606961 20055 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:24.607442 20055 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
W20260808 10:43:24.608234 19637 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
W20260808 10:43:24.608345 19635 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
I20260808 10:43:24.608465 19635 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0534a64ea264cfbac23bea2636b8a24; no voters: 4027456275494db3a5aa5323b1e40c09, 6d7191491dbf49c28e0470c66c345217
I20260808 10:43:24.608628 20055 raft_consensus.cc:2726] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:43:24.627830 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:24.627437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:58289)
0808 10:43:24.627678 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:24.627736 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260808 10:43:25.306564 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:25.306131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:46629)
0808 10:43:25.306392 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:25.306455 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260808 10:43:25.432542 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:25.432032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:44123)
0808 10:43:25.432368 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:25.432438 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":79,"threads_started":1}
I20260808 10:43:25.629192 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:25.628691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:58513)
0808 10:43:25.629018 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:25.629094 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":92,"threads_started":1}
I20260808 10:43:26.201987 20060 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:26.202134 20060 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:26.202867 20060 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
W20260808 10:43:26.203464 19637 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
W20260808 10:43:26.203572 19635 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
I20260808 10:43:26.203655 19635 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0534a64ea264cfbac23bea2636b8a24; no voters: 4027456275494db3a5aa5323b1e40c09, 6d7191491dbf49c28e0470c66c345217
I20260808 10:43:26.204129 20060 raft_consensus.cc:2726] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:43:26.307839 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:26.307407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:33591)
0808 10:43:26.307689 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:26.307745 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260808 10:43:26.434016 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:26.433467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:49965)
0808 10:43:26.433846 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:26.433919 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":146,"threads_started":1}
I20260808 10:43:26.630497 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:26.630022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:60717)
0808 10:43:26.630321 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:26.630383 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":97,"threads_started":1}
I20260808 10:43:27.309329 20064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:27.308779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:43339)
0808 10:43:27.309128 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:27.309193 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":101,"threads_started":1}
I20260808 10:43:27.435317 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:27.434861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:54757)
0808 10:43:27.435153 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:27.435219 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
I20260808 10:43:27.631695 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:27.631280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:36665)
0808 10:43:27.631536 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:27.631591 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260808 10:43:28.310806 20067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:28.310301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:41881)
0808 10:43:28.310639 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:28.310706 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":75,"threads_started":1}
I20260808 10:43:28.436748 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:28.436212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:51359)
0808 10:43:28.436556 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:28.436628 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":79,"threads_started":1}
I20260808 10:43:28.632987 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:28.632503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:37787)
0808 10:43:28.632802 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:28.632872 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
I20260808 10:43:29.312397 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:29.311882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:53211)
0808 10:43:29.312213 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:29.312290 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
I20260808 10:43:29.438177 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:29.437696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:56993)
0808 10:43:29.438014 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:29.438082 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":82,"threads_started":1}
I20260808 10:43:29.634445 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:29.633978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:37291)
0808 10:43:29.634278 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:29.634347 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260808 10:43:30.313894 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:30.313303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:44789)
0808 10:43:30.313672 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:30.313755 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":102,"threads_started":1}
I20260808 10:43:30.439597 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:30.439032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:38459)
0808 10:43:30.439353 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:30.439430 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":123,"threads_started":1}
I20260808 10:43:30.635813 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:30.635320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:60225)
0808 10:43:30.635655 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:30.635718 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260808 10:43:31.315136 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:31.314705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:48649)
0808 10:43:31.314975 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:31.315036 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260808 10:43:31.326660 20079 raft_consensus.cc:482] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:31.326794 20079 raft_consensus.cc:504] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4027456275494db3a5aa5323b1e40c09" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44835 } } peers { permanent_uuid: "6d7191491dbf49c28e0470c66c345217" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34851 } } peers { permanent_uuid: "e0534a64ea264cfbac23bea2636b8a24" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37473 } }
I20260808 10:43:31.327286 20079 leader_election.cc:288] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835), 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851)
W20260808 10:43:31.328034 19635 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d7191491dbf49c28e0470c66c345217 (127.1.194.66:34851): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
W20260808 10:43:31.328162 19637 leader_election.cc:341] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4027456275494db3a5aa5323b1e40c09 (127.1.194.67:44835): Not found: Tablet not found: 6cca9fb0060a452da7694594957c7826
I20260808 10:43:31.328286 19637 leader_election.cc:302] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0534a64ea264cfbac23bea2636b8a24; no voters: 4027456275494db3a5aa5323b1e40c09, 6d7191491dbf49c28e0470c66c345217
I20260808 10:43:31.328445 20079 raft_consensus.cc:2726] T 6cca9fb0060a452da7694594957c7826 P e0534a64ea264cfbac23bea2636b8a24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:43:31.440857 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:31.440427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:56407)
0808 10:43:31.440690 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:31.440755 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260808 10:43:31.637162 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:31.636703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:52435)
0808 10:43:31.636994 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:31.637061 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
I20260808 10:43:32.316551 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:32.316048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:46795)
0808 10:43:32.316370 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:32.316450 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":87,"threads_started":1}
I20260808 10:43:32.442369 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:32.441878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:50263)
0808 10:43:32.442190 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:32.442253 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":88,"threads_started":1}
I20260808 10:43:32.638412 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:32.637965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.67:54669)
0808 10:43:32.638254 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:32.638316 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":73,"threads_started":1}
I20260808 10:43:33.318090 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:33.317658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.65:57215)
0808 10:43:33.317934 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:33.317995 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260808 10:43:33.443864 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:33.443388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:37735 (local address 127.1.194.66:41297)
0808 10:43:33.443709 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:33.443769 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.126:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
I20260808 10:43:33.619441 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 19571
I20260808 10:43:33.627162 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 19757
I20260808 10:43:33.634099 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 19893
2026-08-08T10:43:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59301 ms)
[----------] 30 tests from SecurityITest (249605 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:33 dist-test-slave-ss4n krb5kdc[20089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:33 dist-test-slave-ss4n krb5kdc[20089](info): set up 2 sockets
Aug 08 10:43:33 dist-test-slave-ss4n krb5kdc[20089](info): commencing operation
krb5kdc: starting...
W20260808 10:43:34.748478 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:34 dist-test-slave-ss4n krb5kdc[20089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:43:34.885182 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45995
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:34.999477 20103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:34.999759 20103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:34.999804 20103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:35.004051 20103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:35.004127 20103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:35.004146 20103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:35.004163 20103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:35.004179 20103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:35.009701 20103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:35.010838 20103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:35.011871 20103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:35.021013 20109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.020999 20108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.021026 20111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:35.021373 20103 server_base.cc:1034] running on GCE node
I20260808 10:43:35.021927 20103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:35.023129 20103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:35.024324 20103 hybrid_clock.cc:648] HybridClock initialized: now 1786185815024315 us; error 31 us; skew 500 ppm
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:35.027231 20103 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:43:35.028252 20103 webserver.cc:460] Webserver started at http://127.1.194.126:38059/ using document root <none> and password file <none>
I20260808 10:43:35.028820 20103 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:35.028911 20103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:35.029150 20103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:35.031213 20103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "1b1d0875954c428982a61ebd3dc034bc"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "176b60c029850d6ffcf69f7917cbd2c4"
server_key_iv: "cf0d5fd83cd350a649f3b2c11986908b"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.031788 20103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "1b1d0875954c428982a61ebd3dc034bc"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "176b60c029850d6ffcf69f7917cbd2c4"
server_key_iv: "cf0d5fd83cd350a649f3b2c11986908b"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.035712 20103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:43:35.038313 20118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.039776 20103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:35.039932 20103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "1b1d0875954c428982a61ebd3dc034bc"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "176b60c029850d6ffcf69f7917cbd2c4"
server_key_iv: "cf0d5fd83cd350a649f3b2c11986908b"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.040055 20103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:35.061728 20103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:35.065162 20103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:35.065380 20103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:35.072818 20103 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:45995
I20260808 10:43:35.072849 20172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:45995 every 8 connection(s)
I20260808 10:43:35.073865 20103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:35.076663 20173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:35.082052 20173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: Bootstrap starting.
I20260808 10:43:35.082809 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20103
I20260808 10:43:35.082953 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:35.083266 1801 external_mini_cluster.cc:1428] Setting key 3d414aea03af2745d6dcb5533de1f8ee
I20260808 10:43:35.085022 20173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:35.085816 20173 log.cc:824] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:35.088133 20173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: No bootstrap required, opened a new log
I20260808 10:43:35.092723 20173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b1d0875954c428982a61ebd3dc034bc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45995 } }
I20260808 10:43:35.092979 20173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:35.093063 20173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b1d0875954c428982a61ebd3dc034bc, State: Initialized, Role: FOLLOWER
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:35.093592 20173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b1d0875954c428982a61ebd3dc034bc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45995 } }
I20260808 10:43:35.093729 20173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:35.093838 20173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:35.093971 20173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:35.094895 20173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b1d0875954c428982a61ebd3dc034bc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45995 } }
I20260808 10:43:35.095312 20173 leader_election.cc:302] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b1d0875954c428982a61ebd3dc034bc; no voters:
I20260808 10:43:35.095674 20173 leader_election.cc:288] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:35.095795 20178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:35.096060 20178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [term 1 LEADER]: Becoming Leader. State: Replica: 1b1d0875954c428982a61ebd3dc034bc, State: Running, Role: LEADER
I20260808 10:43:35.096457 20178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b1d0875954c428982a61ebd3dc034bc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45995 } }
I20260808 10:43:35.097101 20173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:35.097780 20178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b1d0875954c428982a61ebd3dc034bc. Latest consensus state: current_term: 1 leader_uuid: "1b1d0875954c428982a61ebd3dc034bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b1d0875954c428982a61ebd3dc034bc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45995 } } }
I20260808 10:43:35.098650 20178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:35.098923 20180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b1d0875954c428982a61ebd3dc034bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b1d0875954c428982a61ebd3dc034bc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45995 } } }
I20260808 10:43:35.100144 20180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:35.100261 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.084513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.1.194.126:45995)
0808 10:43:35.084945 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.084960 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.084995 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:43:35.085729 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.085749 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.086127 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.086564 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:35.087631 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.088657 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.089888 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.090382 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.090509 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.093879 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:35.093883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:35.093896 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:35.093944 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:35.096148 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.096724 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.096730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.096736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.096890 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.097236 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.097239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.097241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.097834 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:35.098771 (+ 937us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.098952 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.099282 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":144,"threads_started":1}
W20260808 10:43:35.102458 20194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:43:35.102558 20194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:43:35.102648 20190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:35.103395 20190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:35.109938 20190 catalog_manager.cc:1347] Generated new cluster ID: a1cc896e22dc4043977fc80913303699
I20260808 10:43:35.110018 20190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:35.123065 20190 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:35.124358 20190 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:35.133467 20190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: Generated new TSK 0
I20260808 10:43:35.134227 20190 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:43:35.169292 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45995
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:35.281842 20200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:35.282141 20200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:35.282239 20200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:35.286326 20200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:35.286417 20200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:35.286509 20200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:35.291790 20200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:35.292934 20200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:35.294123 20200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:35.304865 20205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.304872 20208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:35.304931 20200 server_base.cc:1034] running on GCE node
W20260808 10:43:35.304865 20206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:35.305744 20200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:35.306404 20200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:35.307574 20200 hybrid_clock.cc:648] HybridClock initialized: now 1786185815307554 us; error 31 us; skew 500 ppm
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:35.310667 20200 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:43:35.311599 20200 webserver.cc:460] Webserver started at http://127.1.194.65:41593/ using document root <none> and password file <none>
I20260808 10:43:35.312170 20200 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:35.312263 20200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:35.312498 20200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:35.314605 20200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "a1b2e5c5b0a1465b9d6a665373c743c2"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "338b15483a6a2d56283e79f67b04477e"
server_key_iv: "6b841e769acd50658b9ef628fa726f38"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.315215 20200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "a1b2e5c5b0a1465b9d6a665373c743c2"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "338b15483a6a2d56283e79f67b04477e"
server_key_iv: "6b841e769acd50658b9ef628fa726f38"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.319077 20200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:43:35.321908 20215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.323230 20200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:35.323387 20200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "a1b2e5c5b0a1465b9d6a665373c743c2"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "338b15483a6a2d56283e79f67b04477e"
server_key_iv: "6b841e769acd50658b9ef628fa726f38"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.323544 20200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:35.345474 20200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:35.349032 20200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:35.349254 20200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:35.349951 20200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:35.350870 20200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:35.350955 20200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.351033 20200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:35.351087 20200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.360431 20200 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:37073
I20260808 10:43:35.360459 20332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:37073 every 8 connection(s)
I20260808 10:43:35.361384 20200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:43:35.367340 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20200
I20260808 10:43:35.367556 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:43:35.367873 1801 external_mini_cluster.cc:1428] Setting key 19a13f621040077c021453dc512e6d54
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:35.376274 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.363452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:34915 (local address 127.1.194.126:45995)
0808 10:43:35.363681 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.363686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.364481 (+ 795us) server_negotiation.cc:407] Connection header received
0808 10:43:35.365362 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.365365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.365411 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.365612 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:35.366789 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.367311 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.368658 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.368980 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.369094 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.371954 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:35.371957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:35.371959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:35.371983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:35.373755 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.374451 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.374455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.374456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.374545 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.374998 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.375001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.375003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.375211 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:35.375298 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.376023 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.376147 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260808 10:43:35.377507 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.363859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45995 (local address 127.1.194.65:34915)
0808 10:43:35.364316 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.364357 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.365098 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.365754 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.365760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.366107 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:35.366642 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.366660 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.367458 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.367461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.368492 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.368501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.369194 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.369201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.369411 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.370242 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:35.370259 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:35.371772 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:35.373888 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:35.373898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:35.373917 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:35.374324 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:35.374655 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:35.374659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:35.374662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:35.374874 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:35.375330 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:35.375339 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:35.375734 (+ 395us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.375981 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.376368 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:43:35.378866 20333 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45995
I20260808 10:43:35.379153 20333 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:35.379655 20333 heartbeater.cc:511] Master 127.1.194.126:45995 requested a full tablet report, sending...
I20260808 10:43:35.381377 20136 ts_manager.cc:194] Registered new tserver with Master: a1b2e5c5b0a1465b9d6a665373c743c2 (127.1.194.65:37073)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
I20260808 10:43:35.382668 20136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:34915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:43:35.401301 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45995
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:35.517582 20338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:35.517830 20338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:35.517874 20338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:35.522056 20338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:35.522117 20338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:35.522190 20338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:35.527602 20338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:35.528607 20338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:35.529547 20338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:35.539271 20347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.539302 20344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.539302 20345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:35.539368 20338 server_base.cc:1034] running on GCE node
I20260808 10:43:35.540067 20338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:35.540836 20338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:35.542037 20338 hybrid_clock.cc:648] HybridClock initialized: now 1786185815542014 us; error 39 us; skew 500 ppm
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:35.545137 20338 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:43:35.546069 20338 webserver.cc:460] Webserver started at http://127.1.194.66:37309/ using document root <none> and password file <none>
I20260808 10:43:35.546761 20338 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:35.546828 20338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:35.547000 20338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:35.549006 20338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "2b3351997df347d293d820c64157ca83"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "d6c240ad24ba6413e78f61ee8cb59bfc"
server_key_iv: "59b0597c6418504340b47d9ed45006b1"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.549625 20338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "2b3351997df347d293d820c64157ca83"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "d6c240ad24ba6413e78f61ee8cb59bfc"
server_key_iv: "59b0597c6418504340b47d9ed45006b1"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.553330 20338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:43:35.555807 20355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.557013 20338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:35.557107 20338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "2b3351997df347d293d820c64157ca83"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "d6c240ad24ba6413e78f61ee8cb59bfc"
server_key_iv: "59b0597c6418504340b47d9ed45006b1"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.557276 20338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:35.587433 20338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:35.590798 20338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:35.591019 20338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:35.591622 20338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:35.592450 20338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:35.592497 20338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.592568 20338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:35.592602 20338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.602097 20338 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:33021
I20260808 10:43:35.602140 20472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:33021 every 8 connection(s)
I20260808 10:43:35.603016 20338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:35.608938 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20338
I20260808 10:43:35.609100 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:43:35.609404 1801 external_mini_cluster.cc:1428] Setting key fce86a870e904e39cda54bc4a69fb1d6
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:35.617182 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.604902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:40543 (local address 127.1.194.126:45995)
0808 10:43:35.605042 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.605045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.605912 (+ 867us) server_negotiation.cc:407] Connection header received
0808 10:43:35.606788 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.606790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.606836 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.606913 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:35.608074 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.608589 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.610118 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.610361 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.610444 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.613328 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:35.613331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:35.613333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:35.613356 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:35.614834 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.615494 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.615497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.615499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.615581 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.616032 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.616036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.616038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.616234 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:35.616315 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.616965 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.617062 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:43:35.618463 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.605282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45995 (local address 127.1.194.66:40543)
0808 10:43:35.605756 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.605798 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.606550 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.607068 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.607074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.607430 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:35.607936 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.607949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.608719 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.608726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.610014 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.610023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.610510 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.610514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.610741 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.611401 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:35.611423 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:35.613131 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:35.614955 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:35.614963 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:35.614980 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:35.615361 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:35.615693 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:35.615699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:35.615704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:35.615903 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:35.616348 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:35.616356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:35.616669 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.616899 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.617288 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":153,"threads_started":1}
I20260808 10:43:35.619705 20473 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45995
I20260808 10:43:35.619946 20473 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:35.620419 20473 heartbeater.cc:511] Master 127.1.194.126:45995 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:43:35.621560 20136 ts_manager.cc:194] Registered new tserver with Master: 2b3351997df347d293d820c64157ca83 (127.1.194.66:33021)
I20260808 10:43:35.622190 20136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:40543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:43:35.645112 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45995
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:35.760967 20478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:35.761284 20478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:35.761366 20478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:35.765625 20478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:35.765688 20478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:35.765777 20478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:35.771030 20478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:35.772076 20478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:35.773098 20478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:35.782879 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.782826 20485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:35.782826 20484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:35.782859 20478 server_base.cc:1034] running on GCE node
I20260808 10:43:35.783496 20478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:35.784216 20478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:35.785429 20478 hybrid_clock.cc:648] HybridClock initialized: now 1786185815785394 us; error 55 us; skew 500 ppm
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:35.788419 20478 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:43:35.789378 20478 webserver.cc:460] Webserver started at http://127.1.194.67:36203/ using document root <none> and password file <none>
I20260808 10:43:35.790004 20478 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:35.790107 20478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:35.790335 20478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:35.792701 20478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "195c8029da124e48a7205a70a58751b3"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "6473e969c07df77cb7c47affec2d23fc"
server_key_iv: "4fbb38483fd6c9d8c21fabcc80a2327f"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.793385 20478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "195c8029da124e48a7205a70a58751b3"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "6473e969c07df77cb7c47affec2d23fc"
server_key_iv: "4fbb38483fd6c9d8c21fabcc80a2327f"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.797456 20478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:43:35.800472 20495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.801777 20478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:35.801928 20478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "195c8029da124e48a7205a70a58751b3"
format_stamp: "Formatted at 2026-08-08 10:43:35 on dist-test-slave-ss4n"
server_key: "6473e969c07df77cb7c47affec2d23fc"
server_key_iv: "4fbb38483fd6c9d8c21fabcc80a2327f"
server_key_version: "encryptionkey@0"
I20260808 10:43:35.802057 20478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:35.834817 20478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:35.838171 20478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:35.838384 20478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:35.838999 20478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:35.839835 20478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:35.839910 20478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.839977 20478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:35.840026 20478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:35.849551 20478 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:45243
I20260808 10:43:35.849627 20611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:45243 every 8 connection(s)
I20260808 10:43:35.850469 20478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:35.852842 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20478
I20260808 10:43:35.852993 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:43:35.853372 1801 external_mini_cluster.cc:1428] Setting key 4e59c343ea57dd569dee50d5c60709d6
Aug 08 10:43:35 dist-test-slave-ss4n krb5kdc[20089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:35.864372 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.852300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:46625 (local address 127.1.194.126:45995)
0808 10:43:35.852437 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.852441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.853426 (+ 985us) server_negotiation.cc:407] Connection header received
0808 10:43:35.854603 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.854606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.854653 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.854754 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:35.856090 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.856622 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.857859 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.858107 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.858188 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.860829 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:35.860832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:35.860840 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:35.860862 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:35.862299 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.862863 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.862869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.862871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.862950 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.863297 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.863301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.863303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.863484 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:35.863593 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.864083 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.864200 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:43:35.865159 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.852673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45995 (local address 127.1.194.67:46625)
0808 10:43:35.853238 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.853298 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.854406 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.854931 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.854940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.855321 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:35.855945 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.855959 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.856732 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.856735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.857759 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.857768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.858251 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.858254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.858393 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.859052 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:35.859068 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:35.860667 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:35.862417 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:35.862423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:35.862437 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:35.862771 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:35.863046 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:35.863052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:35.863056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:35.863197 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:35.863594 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:35.863601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:35.863840 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.864030 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.864340 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":143,"threads_started":1}
I20260808 10:43:35.866410 20612 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45995
I20260808 10:43:35.866688 20612 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:35.867270 20612 heartbeater.cc:511] Master 127.1.194.126:45995 requested a full tablet report, sending...
I20260808 10:43:35.868266 20136 ts_manager.cc:194] Registered new tserver with Master: 195c8029da124e48a7205a70a58751b3 (127.1.194.67:45243)
I20260808 10:43:35.868865 20136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:46625
I20260808 10:43:35.879675 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:43:35.889362 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.881795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49500 (local address 127.1.194.126:45995)
0808 10:43:35.881942 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.881946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.882049 (+ 103us) server_negotiation.cc:407] Connection header received
0808 10:43:35.882140 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.882143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.882197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.882277 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:35.883025 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.883544 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.884382 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.884636 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.884735 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.886202 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:35.886205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:35.886207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:35.886231 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:35.887822 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.888227 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.888230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.888231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.888322 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:35.888590 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:35.888593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:35.888594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:35.888834 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:35.888953 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.889144 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.889240 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:43:35.892442 20136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:43:35.895965 20136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:43:35.912171 20629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.906194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45243 (local address 127.0.0.1:44606)
0808 10:43:35.906925 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.906940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.907087 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.907767 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.907771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.907789 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:35.907887 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.907893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.909470 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.909474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.910634 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.910647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.911597 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.911601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.911709 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.911746 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.911860 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.911998 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":51,"thread_start_us":121,"threads_started":1}
I20260808 10:43:35.912173 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.905367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37073 (local address 127.0.0.1:59206)
0808 10:43:35.905963 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.906018 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.906144 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.907034 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.907041 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.907068 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:35.907223 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.907231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.908970 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.908976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.910500 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.910512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.911829 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.911835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.911938 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.911958 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.912031 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.912088 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":105,"threads_started":1}
I20260808 10:43:35.912843 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.906862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57802 (local address 127.1.194.66:33021)
0808 10:43:35.907476 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.907489 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.907513 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:43:35.907583 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.907587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.907712 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.907878 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:35.908141 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.908875 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.910929 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.911821 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.911930 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.912086 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.912566 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.912684 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":156,"threads_started":1}
I20260808 10:43:35.912380 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.905676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.1.194.65:37073)
0808 10:43:35.906100 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.906108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.906136 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:43:35.906435 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.906442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.906680 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.906843 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:35.907435 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.908822 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.910654 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.911578 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.911704 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.911857 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.912094 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.912203 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":118,"threads_started":1}
I20260808 10:43:35.912971 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.905802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.1.194.67:45243)
0808 10:43:35.906506 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.906524 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.907089 (+ 565us) server_negotiation.cc:407] Connection header received
0808 10:43:35.907202 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.907207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.907412 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.907571 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:35.908042 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.909315 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.910774 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.911428 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.911574 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.911746 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.912254 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.912830 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":158,"threads_started":1}
I20260808 10:43:35.913390 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.906572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:33021 (local address 127.0.0.1:57802)
0808 10:43:35.907299 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.907312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.907409 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.907861 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.907864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.907882 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:35.908009 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.908020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.909087 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.909090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.910798 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.910807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.912157 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.912161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.912254 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.912280 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.912357 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.912443 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":539,"thread_start_us":112,"threads_started":1}
I20260808 10:43:35.916261 20267 tablet_service.cc:1467] Processing CreateTablet for tablet 231e008cbfdd4d099a67a24c0f50f51d (DEFAULT_TABLE table=test-table [id=e5f583cd73484ad6b033cd92d9666c94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:35.916263 20406 tablet_service.cc:1467] Processing CreateTablet for tablet 231e008cbfdd4d099a67a24c0f50f51d (DEFAULT_TABLE table=test-table [id=e5f583cd73484ad6b033cd92d9666c94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:35.917521 20406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 231e008cbfdd4d099a67a24c0f50f51d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:35.917657 20267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 231e008cbfdd4d099a67a24c0f50f51d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:35.920960 20545 tablet_service.cc:1467] Processing CreateTablet for tablet 231e008cbfdd4d099a67a24c0f50f51d (DEFAULT_TABLE table=test-table [id=e5f583cd73484ad6b033cd92d9666c94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:35.922240 20545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 231e008cbfdd4d099a67a24c0f50f51d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:35.925251 20631 tablet_bootstrap.cc:492] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Bootstrap starting.
I20260808 10:43:35.925928 20632 tablet_bootstrap.cc:492] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2: Bootstrap starting.
I20260808 10:43:35.928283 20632 tablet_bootstrap.cc:654] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:35.928288 20631 tablet_bootstrap.cc:654] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:35.929348 20632 log.cc:824] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:35.929417 20631 log.cc:824] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:35.929656 20633 tablet_bootstrap.cc:492] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Bootstrap starting.
I20260808 10:43:35.931399 20631 tablet_bootstrap.cc:492] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: No bootstrap required, opened a new log
I20260808 10:43:35.931629 20631 ts_tablet_manager.cc:1397] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:43:35.932018 20633 tablet_bootstrap.cc:654] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:35.932067 20632 tablet_bootstrap.cc:492] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2: No bootstrap required, opened a new log
I20260808 10:43:35.932245 20632 ts_tablet_manager.cc:1397] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:43:35.933928 20633 log.cc:824] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:35.935256 20631 raft_consensus.cc:348] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.935513 20631 raft_consensus.cc:374] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:35.935777 20631 raft_consensus.cc:729] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b3351997df347d293d820c64157ca83, State: Initialized, Role: FOLLOWER
I20260808 10:43:35.936094 20633 tablet_bootstrap.cc:492] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: No bootstrap required, opened a new log
I20260808 10:43:35.936290 20633 ts_tablet_manager.cc:1397] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:43:35.936296 20631 consensus_queue.cc:260] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.936950 20473 heartbeater.cc:503] Master 127.1.194.126:45995 was elected leader, sending a full tablet report...
I20260808 10:43:35.937180 20632 raft_consensus.cc:348] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.937445 20632 raft_consensus.cc:374] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:35.937585 20632 raft_consensus.cc:729] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b2e5c5b0a1465b9d6a665373c743c2, State: Initialized, Role: FOLLOWER
I20260808 10:43:35.938043 20632 consensus_queue.cc:260] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.938406 20631 ts_tablet_manager.cc:1428] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:43:35.939060 20632 ts_tablet_manager.cc:1428] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 10:43:35.940093 20333 heartbeater.cc:503] Master 127.1.194.126:45995 was elected leader, sending a full tablet report...
I20260808 10:43:35.942961 20633 raft_consensus.cc:348] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.943251 20633 raft_consensus.cc:374] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:35.943343 20633 raft_consensus.cc:729] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195c8029da124e48a7205a70a58751b3, State: Initialized, Role: FOLLOWER
I20260808 10:43:35.943807 20633 consensus_queue.cc:260] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.944516 20612 heartbeater.cc:503] Master 127.1.194.126:45995 was elected leader, sending a full tablet report...
I20260808 10:43:35.944871 20633 ts_tablet_manager.cc:1428] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260808 10:43:35.987922 20637 raft_consensus.cc:482] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:35.988171 20637 raft_consensus.cc:504] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.989379 20637 leader_election.cc:288] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b2e5c5b0a1465b9d6a665373c743c2 (127.1.194.65:37073), 195c8029da124e48a7205a70a58751b3 (127.1.194.67:45243)
I20260808 10:43:35.993335 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.989753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:53029 (local address 127.1.194.65:37073)
0808 10:43:35.989905 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.989913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.989931 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:43:35.990018 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.990025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.990089 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.990201 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:35.990515 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.991077 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.992232 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.992701 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.992846 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.992903 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.993122 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.993203 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:43:35.993337 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.989589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37073 (local address 127.1.194.66:53029)
0808 10:43:35.989732 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.989749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.989848 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.990215 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.990219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.990254 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:35.990379 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.990385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.991206 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.991210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.992077 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.992087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.992838 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.992841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.992964 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.993002 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.993104 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.993189 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":2,"spinlock_wait_cycles":4224}
I20260808 10:43:35.993441 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.989906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:43583 (local address 127.1.194.67:45243)
0808 10:43:35.990035 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:35.990039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:35.990202 (+ 163us) server_negotiation.cc:407] Connection header received
0808 10:43:35.990364 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:35.990367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:35.990412 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:35.990505 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:35.990776 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.991330 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.992410 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.992844 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.992978 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.993036 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:35.993252 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 10:43:35.993318 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:43:35.993530 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:35.989750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:45243 (local address 127.1.194.66:43583)
0808 10:43:35.990107 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:35.990118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:35.990252 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:35.990517 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:35.990520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:35.990537 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:35.990665 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.990671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.991479 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.991483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.992281 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:35.992289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:35.992972 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:35.992976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:35.993112 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:35.993142 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 10:43:35.993235 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 10:43:35.993303 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":23,"spinlock_wait_cycles":6400,"thread_start_us":142,"threads_started":1}
I20260808 10:43:35.994240 20287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231e008cbfdd4d099a67a24c0f50f51d" candidate_uuid: "2b3351997df347d293d820c64157ca83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" is_pre_election: true
I20260808 10:43:35.994232 20565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231e008cbfdd4d099a67a24c0f50f51d" candidate_uuid: "2b3351997df347d293d820c64157ca83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195c8029da124e48a7205a70a58751b3" is_pre_election: true
I20260808 10:43:35.994549 20287 raft_consensus.cc:2454] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b3351997df347d293d820c64157ca83 in term 0.
I20260808 10:43:35.994566 20565 raft_consensus.cc:2454] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b3351997df347d293d820c64157ca83 in term 0.
I20260808 10:43:35.995071 20357 leader_election.cc:302] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b3351997df347d293d820c64157ca83, a1b2e5c5b0a1465b9d6a665373c743c2; no voters:
I20260808 10:43:35.995379 20637 raft_consensus.cc:2781] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:43:35.995488 20637 raft_consensus.cc:482] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:43:35.995537 20637 raft_consensus.cc:3037] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:35.996580 20637 raft_consensus.cc:504] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:35.996995 20637 leader_election.cc:288] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [CANDIDATE]: Term 1 election: Requested vote from peers a1b2e5c5b0a1465b9d6a665373c743c2 (127.1.194.65:37073), 195c8029da124e48a7205a70a58751b3 (127.1.194.67:45243)
I20260808 10:43:35.997390 20287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231e008cbfdd4d099a67a24c0f50f51d" candidate_uuid: "2b3351997df347d293d820c64157ca83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2"
I20260808 10:43:35.997427 20565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231e008cbfdd4d099a67a24c0f50f51d" candidate_uuid: "2b3351997df347d293d820c64157ca83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195c8029da124e48a7205a70a58751b3"
I20260808 10:43:35.997573 20287 raft_consensus.cc:3037] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:35.997649 20565 raft_consensus.cc:3037] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:35.998771 20287 raft_consensus.cc:2454] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b3351997df347d293d820c64157ca83 in term 1.
I20260808 10:43:35.998828 20565 raft_consensus.cc:2454] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b3351997df347d293d820c64157ca83 in term 1.
I20260808 10:43:35.999122 20357 leader_election.cc:302] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b3351997df347d293d820c64157ca83, a1b2e5c5b0a1465b9d6a665373c743c2; no voters:
I20260808 10:43:35.999289 20637 raft_consensus.cc:2781] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:35.999549 20637 raft_consensus.cc:686] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 1 LEADER]: Becoming Leader. State: Replica: 2b3351997df347d293d820c64157ca83, State: Running, Role: LEADER
I20260808 10:43:35.999903 20637 consensus_queue.cc:237] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } }
I20260808 10:43:36.003202 20135 catalog_manager.cc:5495] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b3351997df347d293d820c64157ca83 (127.1.194.66). New cstate: current_term: 1 leader_uuid: "2b3351997df347d293d820c64157ca83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b3351997df347d293d820c64157ca83" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33021 } health_report { overall_health: HEALTHY } } }
I20260808 10:43:36.016103 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.012292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.1.194.66:33021)
0808 10:43:36.012450 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.012463 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.012477 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:43:36.012641 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.012643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.012699 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.012816 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:43:36.013066 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.013655 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.014592 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.014815 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.014928 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.015247 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:43:36.015686 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:43:36.015826 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:36.015889 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:43:36.015949 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:43:36.022280 20287 raft_consensus.cc:1261] T 231e008cbfdd4d099a67a24c0f50f51d P a1b2e5c5b0a1465b9d6a665373c743c2 [term 1 FOLLOWER]: Refusing update from remote peer 2b3351997df347d293d820c64157ca83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:36.022280 20565 raft_consensus.cc:1261] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 1 FOLLOWER]: Refusing update from remote peer 2b3351997df347d293d820c64157ca83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:36.023007 20641 consensus_queue.cc:1035] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b2e5c5b0a1465b9d6a665373c743c2" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:36.023263 20637 consensus_queue.cc:1035] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "195c8029da124e48a7205a70a58751b3" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 45243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:36.030308 20644 mvcc.cc:205] Tried to move back new op lower bound from 7316217102419091456 to 7316217102338363392. Current Snapshot: MvccSnapshot[applied={T|T < 7316217102419091456}]
I20260808 10:43:36.032729 20646 mvcc.cc:205] Tried to move back new op lower bound from 7316217102419091456 to 7316217102338363392. Current Snapshot: MvccSnapshot[applied={T|T < 7316217102419091456}]
I20260808 10:43:36.033028 20645 mvcc.cc:205] Tried to move back new op lower bound from 7316217102419091456 to 7316217102338363392. Current Snapshot: MvccSnapshot[applied={T|T < 7316217102419091456}]
I20260808 10:43:36.040477 20134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:36.040721 20134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:36.043346 20134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b1d0875954c428982a61ebd3dc034bc: Sending DeleteTablet for 3 replicas of tablet 231e008cbfdd4d099a67a24c0f50f51d
I20260808 10:43:36.044286 20406 tablet_service.cc:1514] Processing DeleteTablet for tablet 231e008cbfdd4d099a67a24c0f50f51d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:57802
I20260808 10:43:36.044284 20267 tablet_service.cc:1514] Processing DeleteTablet for tablet 231e008cbfdd4d099a67a24c0f50f51d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:59206
I20260808 10:43:36.044421 20545 tablet_service.cc:1514] Processing DeleteTablet for tablet 231e008cbfdd4d099a67a24c0f50f51d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:36 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:44606
I20260808 10:43:36.045019 20656 tablet_replica.cc:330] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: stopping tablet replica
I20260808 10:43:36.045122 20654 tablet_replica.cc:330] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: stopping tablet replica
I20260808 10:43:36.045538 20654 raft_consensus.cc:2229] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:43:36.045634 20656 raft_consensus.cc:2229] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:36.045943 20654 raft_consensus.cc:2258] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:36.045984 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20200
I20260808 10:43:36.046025 20656 raft_consensus.cc:2258] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:36.050375 20656 ts_tablet_manager.cc:1905] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:36.053036 20656 ts_tablet_manager.cc:1918] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:36.053158 20656 log.cc:1196] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/231e008cbfdd4d099a67a24c0f50f51d
I20260808 10:43:36.053555 20656 ts_tablet_manager.cc:1939] T 231e008cbfdd4d099a67a24c0f50f51d P 195c8029da124e48a7205a70a58751b3: Deleting consensus metadata
I20260808 10:43:36.054098 20654 ts_tablet_manager.cc:1905] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:36.054876 20122 catalog_manager.cc:4841] TS 195c8029da124e48a7205a70a58751b3 (127.1.194.67:45243): tablet 231e008cbfdd4d099a67a24c0f50f51d (table test-table [id=e5f583cd73484ad6b033cd92d9666c94]) successfully deleted
I20260808 10:43:36.057454 20654 ts_tablet_manager.cc:1918] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:36.057577 20654 log.cc:1196] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/231e008cbfdd4d099a67a24c0f50f51d
I20260808 10:43:36.057893 20654 ts_tablet_manager.cc:1939] T 231e008cbfdd4d099a67a24c0f50f51d P 2b3351997df347d293d820c64157ca83: Deleting consensus metadata
I20260808 10:43:36.059125 20122 catalog_manager.cc:4841] TS 2b3351997df347d293d820c64157ca83 (127.1.194.66:33021): tablet 231e008cbfdd4d099a67a24c0f50f51d (table test-table [id=e5f583cd73484ad6b033cd92d9666c94]) successfully deleted
W20260808 10:43:36.062001 20120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:37073 (error 108)
I20260808 10:43:36.062860 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20338
I20260808 10:43:36.063149 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.062830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37073 (local address 127.0.0.1:59222)
0808 10:43:36.062969 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.063042 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:43:36.064879 20120 catalog_manager.cc:4584] TS a1b2e5c5b0a1465b9d6a665373c743c2 (127.1.194.65:37073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 231e008cbfdd4d099a67a24c0f50f51d: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37073: connect: Connection refused (error 111)
I20260808 10:43:36.069802 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20478
I20260808 10:43:36.077289 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20103
2026-08-08T10:43:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2442 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T10:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:36Z Disabled control of system clock
I20260808 10:43:36.110678 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:36481
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:43:36.224685 20661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:36.224973 20661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:36.225060 20661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:36.229431 20661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:36.229519 20661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:36.229537 20661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:36.229557 20661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:36.229573 20661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:36.234463 20661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:36481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:36481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:36.235548 20661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:36.237021 20661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:36.245759 20670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.245735 20667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:36.245798 20661 server_base.cc:1034] running on GCE node
W20260808 10:43:36.245723 20668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:36.246793 20661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:36.247893 20661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:36.249068 20661 hybrid_clock.cc:648] HybridClock initialized: now 1786185816249049 us; error 33 us; skew 500 ppm
I20260808 10:43:36.251114 20661 webserver.cc:460] Webserver started at http://127.1.194.126:44207/ using document root <none> and password file <none>
I20260808 10:43:36.251734 20661 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:36.251799 20661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:36.252169 20661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:36.254354 20661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "b0e1338fb14d402390382653624aaedc"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "73c6c2ac85ccb10c218d2e89fede6de4"
server_key_iv: "f637eb416c7a725df9ece9db2a1da1c5"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.254923 20661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "b0e1338fb14d402390382653624aaedc"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "73c6c2ac85ccb10c218d2e89fede6de4"
server_key_iv: "f637eb416c7a725df9ece9db2a1da1c5"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.258584 20661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:43:36.260825 20677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.262079 20661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:36.262233 20661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "b0e1338fb14d402390382653624aaedc"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "73c6c2ac85ccb10c218d2e89fede6de4"
server_key_iv: "f637eb416c7a725df9ece9db2a1da1c5"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.262347 20661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:36.283483 20661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:36.284226 20661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:36.284436 20661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:36.291785 20661 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:36481
I20260808 10:43:36.291800 20731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:36481 every 8 connection(s)
I20260808 10:43:36.292826 20661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:36.295982 20732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:36.297366 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20661
I20260808 10:43:36.297530 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:36.297892 1801 external_mini_cluster.cc:1428] Setting key 59ece886afe69b260ba704a3d4f447ce
I20260808 10:43:36.302510 20732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc: Bootstrap starting.
I20260808 10:43:36.304826 20732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:36.305672 20732 log.cc:824] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:36.307485 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.299331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.1.194.126:36481)
0808 10:43:36.300108 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.300125 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.300163 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:43:36.300891 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.300911 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.301367 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.301750 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:36.302187 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.303218 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.304126 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.304619 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.304793 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.305744 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:36.305750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:36.305764 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:36.306195 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:36.306253 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:36.306274 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:36.306385 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:36.306507 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 10:43:36.306780 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":191,"threads_started":1}
I20260808 10:43:36.307765 20732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc: No bootstrap required, opened a new log
I20260808 10:43:36.311795 20732 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0e1338fb14d402390382653624aaedc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36481 } }
I20260808 10:43:36.312057 20732 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:36.312143 20732 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e1338fb14d402390382653624aaedc, State: Initialized, Role: FOLLOWER
I20260808 10:43:36.312610 20732 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0e1338fb14d402390382653624aaedc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36481 } }
I20260808 10:43:36.312765 20732 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:36.312853 20732 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:36.312955 20732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:36.314013 20732 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0e1338fb14d402390382653624aaedc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36481 } }
I20260808 10:43:36.314355 20732 leader_election.cc:302] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0e1338fb14d402390382653624aaedc; no voters:
I20260808 10:43:36.314781 20732 leader_election.cc:288] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:36.315086 20737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:36.315313 20737 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [term 1 LEADER]: Becoming Leader. State: Replica: b0e1338fb14d402390382653624aaedc, State: Running, Role: LEADER
I20260808 10:43:36.315724 20737 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0e1338fb14d402390382653624aaedc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36481 } }
I20260808 10:43:36.316087 20732 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:36.317366 20739 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0e1338fb14d402390382653624aaedc. Latest consensus state: current_term: 1 leader_uuid: "b0e1338fb14d402390382653624aaedc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0e1338fb14d402390382653624aaedc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36481 } } }
I20260808 10:43:36.317526 20739 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:36.317832 20738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0e1338fb14d402390382653624aaedc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0e1338fb14d402390382653624aaedc" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 36481 } } }
I20260808 10:43:36.317946 20738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:36.318032 20746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:36.320946 20746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:36.327004 20746 catalog_manager.cc:1347] Generated new cluster ID: 2ee440cfe1924076a4a79eca21e6c440
I20260808 10:43:36.327095 20746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:36.336030 20746 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:36.337030 20746 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:36.347271 20746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc: Generated new TSK 0
I20260808 10:43:36.348177 20746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:36.359762 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36481
--builtin_ntp_servers=127.1.194.84:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:43:36.474332 20756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:36.474606 20756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:36.474646 20756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:36.478554 20756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:36.478600 20756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:36.478652 20756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:36.483299 20756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:36.484287 20756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:36.485611 20756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:36.495050 20761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.495076 20762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.495245 20764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:36.495592 20756 server_base.cc:1034] running on GCE node
I20260808 10:43:36.495993 20756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:36.496620 20756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:36.497802 20756 hybrid_clock.cc:648] HybridClock initialized: now 1786185816497782 us; error 29 us; skew 500 ppm
I20260808 10:43:36.499713 20756 webserver.cc:460] Webserver started at http://127.1.194.65:37413/ using document root <none> and password file <none>
I20260808 10:43:36.500291 20756 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:36.500388 20756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:36.500627 20756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:36.503578 20756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "7d132ae4b0454b238bcf88aaf1fcabfa"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "3490bf739f56c6847a9a94fcc7027537"
server_key_iv: "1b7080201ce8cc3a10196d94ddb638d8"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.504374 20756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "7d132ae4b0454b238bcf88aaf1fcabfa"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "3490bf739f56c6847a9a94fcc7027537"
server_key_iv: "1b7080201ce8cc3a10196d94ddb638d8"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.510767 20756 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260808 10:43:36.513370 20770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.514724 20756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:36.514869 20756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "7d132ae4b0454b238bcf88aaf1fcabfa"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "3490bf739f56c6847a9a94fcc7027537"
server_key_iv: "1b7080201ce8cc3a10196d94ddb638d8"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.515012 20756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:36.527786 20756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:36.528465 20756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:36.528669 20756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:36.529260 20756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:36.530244 20756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:36.530321 20756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.530390 20756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:36.530440 20756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.539642 20756 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:39379
I20260808 10:43:36.539664 20892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:39379 every 8 connection(s)
I20260808 10:43:36.540511 20756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:43:36.547047 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20756
I20260808 10:43:36.547185 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:43:36.547519 1801 external_mini_cluster.cc:1428] Setting key 1eba9559b57cecae50b0bed6ed285f1d
I20260808 10:43:36.549898 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36481
--builtin_ntp_servers=127.1.194.84:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:43:36.550858 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.542387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:33095 (local address 127.1.194.126:36481)
0808 10:43:36.542638 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.542644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.543668 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 10:43:36.545024 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.545027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.545079 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.545160 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:36.546150 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.546794 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.548396 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.548779 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.548887 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.550330 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:36.550333 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:36.550335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:36.550348 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:36.550370 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:36.550379 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:36.550444 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:36.550672 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 10:43:36.550739 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":72}
I20260808 10:43:36.552076 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.542873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:36481 (local address 127.1.194.65:33095)
0808 10:43:36.543513 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.543553 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:36.544829 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:36.545316 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:36.545322 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:36.545403 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:36.545990 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.546004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.546946 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.546951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:36.548245 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.548256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.548959 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.548964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:36.549157 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.549979 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:36.549981 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:36.550045 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:36.550148 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:36.550469 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:36.550482 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:36.550491 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:43:36.550634 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 10:43:36.550954 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":117,"threads_started":1}
I20260808 10:43:36.553503 20893 heartbeater.cc:347] Connected to a master server at 127.1.194.126:36481
I20260808 10:43:36.553771 20893 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:36.554440 20893 heartbeater.cc:511] Master 127.1.194.126:36481 requested a full tablet report, sending...
I20260808 10:43:36.556139 20694 ts_manager.cc:194] Registered new tserver with Master: 7d132ae4b0454b238bcf88aaf1fcabfa (127.1.194.65:39379)
I20260808 10:43:36.557300 20694 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.65:33095
W20260808 10:43:36.669797 20896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:36.670070 20896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:36.670146 20896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:36.674173 20896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:36.674238 20896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:36.674358 20896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:36.679313 20896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:36.680450 20896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:36.682235 20896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:36.692301 20902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.692301 20901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.692320 20904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:36.692894 20896 server_base.cc:1034] running on GCE node
I20260808 10:43:36.693248 20896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:36.693941 20896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:36.695185 20896 hybrid_clock.cc:648] HybridClock initialized: now 1786185816695153 us; error 46 us; skew 500 ppm
I20260808 10:43:36.697161 20896 webserver.cc:460] Webserver started at http://127.1.194.66:40753/ using document root <none> and password file <none>
I20260808 10:43:36.697800 20896 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:36.697867 20896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:36.698086 20896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:36.700296 20896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "472648f7982153153052af11e14b1caa"
server_key_iv: "a8870803cd42821fdf26353a79670841"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.700904 20896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "472648f7982153153052af11e14b1caa"
server_key_iv: "a8870803cd42821fdf26353a79670841"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.704913 20896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:43:36.707603 20910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.708927 20896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:43:36.709084 20896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "472648f7982153153052af11e14b1caa"
server_key_iv: "a8870803cd42821fdf26353a79670841"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.709216 20896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:36.728785 20896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:36.729637 20896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:36.729867 20896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:36.730538 20896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:36.731583 20896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:36.731658 20896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.731729 20896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:36.731767 20896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.741465 20896 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:35665
I20260808 10:43:36.741470 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:35665 every 8 connection(s)
I20260808 10:43:36.742487 20896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:36.749058 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 20896
I20260808 10:43:36.749233 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:43:36.749601 1801 external_mini_cluster.cc:1428] Setting key 6d0c62ddb20b793f1a78853bcb613680
I20260808 10:43:36.751853 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36481
--builtin_ntp_servers=127.1.194.84:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:43:36.752485 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.744401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:43161 (local address 127.1.194.126:36481)
0808 10:43:36.744646 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.744650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.745729 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 10:43:36.747164 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.747168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.747229 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.747339 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:36.748223 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.748758 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.750176 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.750454 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.750566 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.751917 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:36.751923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:36.751925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:36.751941 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:36.751978 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:36.751993 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:36.752073 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:36.752278 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 10:43:36.752344 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":131}
I20260808 10:43:36.753676 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.744790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:36481 (local address 127.1.194.66:43161)
0808 10:43:36.745447 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.745624 (+ 177us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:36.746945 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:36.747521 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:36.747527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:36.747609 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:36.748078 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.748089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.748892 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.748895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:36.750046 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.750056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.750637 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.750641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:36.750797 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.751637 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:36.751638 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:36.751680 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:36.751748 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:36.752121 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:36.752133 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:36.752142 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:43:36.752268 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 10:43:36.752582 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":120,"threads_started":1}
I20260808 10:43:36.755121 21028 heartbeater.cc:347] Connected to a master server at 127.1.194.126:36481
I20260808 10:43:36.755431 21028 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:36.756122 21028 heartbeater.cc:511] Master 127.1.194.126:36481 requested a full tablet report, sending...
I20260808 10:43:36.757387 20694 ts_manager.cc:194] Registered new tserver with Master: 1c15bfc18fcd419a9ba2dd46e9a2529b (127.1.194.66:35665)
I20260808 10:43:36.758369 20694 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.66:43161
W20260808 10:43:36.871315 21031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:36.871569 21031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:36.871606 21031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:36.875530 21031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:36.875587 21031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:36.875648 21031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:36.880336 21031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:36481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:36.881368 21031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:36.882802 21031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:36.892505 21036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.892681 21039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:36.892525 21037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:36.892762 21031 server_base.cc:1034] running on GCE node
I20260808 10:43:36.893244 21031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:36.893908 21031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:36.895126 21031 hybrid_clock.cc:648] HybridClock initialized: now 1786185816895109 us; error 26 us; skew 500 ppm
I20260808 10:43:36.897377 21031 webserver.cc:460] Webserver started at http://127.1.194.67:35837/ using document root <none> and password file <none>
I20260808 10:43:36.898140 21031 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:36.898262 21031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:36.898512 21031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:36.900724 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "a38e094c379bc24996753d0805473851"
server_key_iv: "af60477b5c0ab7e059bc79a470d01bea"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.901316 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "a38e094c379bc24996753d0805473851"
server_key_iv: "af60477b5c0ab7e059bc79a470d01bea"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.905097 21031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:43:36.907944 21045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.909276 21031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:43:36.909391 21031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35"
format_stamp: "Formatted at 2026-08-08 10:43:36 on dist-test-slave-ss4n"
server_key: "a38e094c379bc24996753d0805473851"
server_key_iv: "af60477b5c0ab7e059bc79a470d01bea"
server_key_version: "encryptionkey@0"
I20260808 10:43:36.909581 21031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:36.938931 21031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:36.939724 21031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:36.939937 21031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:36.940622 21031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:36.941638 21031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:36.941686 21031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.941756 21031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:36.941789 21031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:36.950778 21031 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:43741
I20260808 10:43:36.950860 21163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:43741 every 8 connection(s)
I20260808 10:43:36.951687 21031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:36.960701 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21031
I20260808 10:43:36.960832 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:43:36.960836 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.953630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:37513 (local address 127.1.194.126:36481)
0808 10:43:36.953780 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.953783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.954847 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 10:43:36.956193 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.956195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.956244 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.956321 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:36.957199 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.957818 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.958944 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.959159 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.959235 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.960344 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:36.960346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:36.960348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:36.960359 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:36.960374 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:36.960380 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:36.960434 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:36.960627 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:43:36.960692 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:43:36.961153 1801 external_mini_cluster.cc:1428] Setting key 89a423661db1e863bc5f17222f6d127b
I20260808 10:43:36.961829 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.953981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:36481 (local address 127.1.194.67:37513)
0808 10:43:36.954685 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.954727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:36.955994 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:36.956478 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:36.956484 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:36.956606 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:36.957072 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.957084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.957935 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.957938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:36.958841 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.958850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.959283 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.959285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:36.959416 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.960116 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:36.960117 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:36.960156 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:36.960216 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:36.960449 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:36.960455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:36.960459 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:43:36.960624 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:43:36.960890 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
I20260808 10:43:36.963057 21164 heartbeater.cc:347] Connected to a master server at 127.1.194.126:36481
I20260808 10:43:36.963296 21164 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:36.963752 21164 heartbeater.cc:511] Master 127.1.194.126:36481 requested a full tablet report, sending...
I20260808 10:43:36.964702 20694 ts_manager.cc:194] Registered new tserver with Master: 3c9a5d1a1ac14a7b90c9f042437a8c35 (127.1.194.67:43741)
I20260808 10:43:36.965274 20694 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.67:37513
I20260808 10:43:36.975044 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:43:36.980650 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.977035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.1.194.126:36481)
0808 10:43:36.977192 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.977197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.977325 (+ 128us) server_negotiation.cc:407] Connection header received
0808 10:43:36.977432 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.977434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.977476 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.977614 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:36.977870 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.978362 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.979160 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.979370 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.979450 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:36.980183 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:36.980186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:36.980188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:36.980198 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:36.980217 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:36.980224 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:36.980281 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:36.980373 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 10:43:36.980420 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:43:36.983592 20694 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:43:36.986578 20694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:43:37.001472 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.995445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.1.194.66:35665)
0808 10:43:36.996286 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.996300 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.996326 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:43:36.996406 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.996413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.996670 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.997204 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:36.997347 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.998708 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.000170 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.000750 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.000882 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.001099 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:37.001215 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 10:43:37.001312 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":130,"threads_started":1}
I20260808 10:43:37.001463 21176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.995286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35665 (local address 127.0.0.1:59804)
0808 10:43:36.995765 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.995813 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:36.995955 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:36.996854 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:36.996860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:36.996884 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:36.997045 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.997056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.998830 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.998833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.000053 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.000065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.000895 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.000898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.001012 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.001051 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:43:37.001166 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:43:37.001296 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":63,"thread_start_us":85,"threads_started":1}
I20260808 10:43:37.001758 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.995992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:43741 (local address 127.0.0.1:51336)
0808 10:43:36.996230 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.996244 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:36.996328 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:36.997382 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:36.997385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:36.997403 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:36.997516 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.997526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.998598 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.998604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.000090 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.000100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.001385 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.001389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.001461 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.001508 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:43:37.001576 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:43:37.001643 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":55,"thread_start_us":104,"threads_started":1}
I20260808 10:43:37.002059 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.995445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39379 (local address 127.0.0.1:41618)
0808 10:43:36.995849 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:36.995876 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:36.995975 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:36.997163 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:36.997166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:36.997183 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:36.997271 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:36.997277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.998867 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:36.998870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.000117 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.000132 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.001180 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.001184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.001269 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.001292 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:43:37.001344 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:43:37.001412 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":110,"thread_start_us":105,"threads_started":1}
I20260808 10:43:37.002319 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.996463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51336 (local address 127.1.194.67:43741)
0808 10:43:36.996926 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.996949 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.996971 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:43:36.997048 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.997053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.997228 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.997392 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:36.997723 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.998481 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.000220 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.001040 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.001157 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.001330 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:37.001910 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 10:43:37.002142 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":170,"threads_started":1}
I20260808 10:43:37.002713 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:36.996090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.1.194.65:39379)
0808 10:43:36.996698 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:36.996709 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:36.996734 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:43:36.996812 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:36.996819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:36.996984 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:36.997413 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:36.997608 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:36.998727 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.000225 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.001040 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.001599 (+ 559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.002130 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:37.002198 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:43:37.002290 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":155,"threads_started":1}
I20260808 10:43:37.004760 21095 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad486cb37b24cf8a3ddc476fda48d60 (DEFAULT_TABLE table=test-table [id=6ee765a5931a496bb48680c38932e2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:37.004827 20962 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad486cb37b24cf8a3ddc476fda48d60 (DEFAULT_TABLE table=test-table [id=6ee765a5931a496bb48680c38932e2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:37.005087 20824 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad486cb37b24cf8a3ddc476fda48d60 (DEFAULT_TABLE table=test-table [id=6ee765a5931a496bb48680c38932e2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:37.005841 21095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad486cb37b24cf8a3ddc476fda48d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:37.005877 20962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad486cb37b24cf8a3ddc476fda48d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:37.005934 20824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad486cb37b24cf8a3ddc476fda48d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:37.011961 21183 tablet_bootstrap.cc:492] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Bootstrap starting.
I20260808 10:43:37.013670 21185 tablet_bootstrap.cc:492] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: Bootstrap starting.
I20260808 10:43:37.014564 21183 tablet_bootstrap.cc:654] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:37.015275 21183 log.cc:824] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:37.015484 21185 tablet_bootstrap.cc:654] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:37.016229 21185 log.cc:824] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:37.016947 21184 tablet_bootstrap.cc:492] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Bootstrap starting.
I20260808 10:43:37.017246 21183 tablet_bootstrap.cc:492] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: No bootstrap required, opened a new log
I20260808 10:43:37.017437 21183 ts_tablet_manager.cc:1397] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:43:37.019068 21184 tablet_bootstrap.cc:654] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:37.019284 21185 tablet_bootstrap.cc:492] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: No bootstrap required, opened a new log
I20260808 10:43:37.019487 21185 ts_tablet_manager.cc:1397] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:43:37.019958 21184 log.cc:824] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:37.021207 21183 raft_consensus.cc:348] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.021437 21183 raft_consensus.cc:374] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:37.021498 21183 raft_consensus.cc:729] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c9a5d1a1ac14a7b90c9f042437a8c35, State: Initialized, Role: FOLLOWER
I20260808 10:43:37.021945 21184 tablet_bootstrap.cc:492] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: No bootstrap required, opened a new log
I20260808 10:43:37.021956 21183 consensus_queue.cc:260] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.022127 21184 ts_tablet_manager.cc:1397] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:37.022620 21164 heartbeater.cc:503] Master 127.1.194.126:36481 was elected leader, sending a full tablet report...
I20260808 10:43:37.022977 21183 ts_tablet_manager.cc:1428] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260808 10:43:37.023967 21185 raft_consensus.cc:348] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.024397 21185 raft_consensus.cc:374] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:37.024456 21185 raft_consensus.cc:729] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d132ae4b0454b238bcf88aaf1fcabfa, State: Initialized, Role: FOLLOWER
I20260808 10:43:37.024855 21185 consensus_queue.cc:260] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.025655 20893 heartbeater.cc:503] Master 127.1.194.126:36481 was elected leader, sending a full tablet report...
I20260808 10:43:37.025800 21185 ts_tablet_manager.cc:1428] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:37.026803 21184 raft_consensus.cc:348] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.027063 21184 raft_consensus.cc:374] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:37.027130 21184 raft_consensus.cc:729] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c15bfc18fcd419a9ba2dd46e9a2529b, State: Initialized, Role: FOLLOWER
I20260808 10:43:37.027546 21184 consensus_queue.cc:260] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.028252 21028 heartbeater.cc:503] Master 127.1.194.126:36481 was elected leader, sending a full tablet report...
I20260808 10:43:37.028491 21184 ts_tablet_manager.cc:1428] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 10:43:37.041741 20894 tablet.cc:2369] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:43:37.084334 21189 raft_consensus.cc:482] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:37.084612 21189 raft_consensus.cc:504] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.085938 21189 leader_election.cc:288] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c15bfc18fcd419a9ba2dd46e9a2529b (127.1.194.66:35665), 7d132ae4b0454b238bcf88aaf1fcabfa (127.1.194.65:39379)
I20260808 10:43:37.090569 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:37.086330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39379 (local address 127.1.194.67:51743)
0808 10:43:37.086752 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:37.086768 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:37.086855 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:37.087068 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:37.087074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:37.087096 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:37.087201 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.087205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.088113 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.088115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.089395 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.089408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.090182 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.090185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.090258 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.090278 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:43:37.090337 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:43:37.090388 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":151,"threads_started":1}
I20260808 10:43:37.090597 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:37.086434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:51743 (local address 127.1.194.65:39379)
0808 10:43:37.086569 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:37.086575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:37.086865 (+ 290us) server_negotiation.cc:407] Connection header received
0808 10:43:37.086925 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:37.086927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:37.086966 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:37.087026 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:37.087300 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.088005 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.089574 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.090052 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.090178 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.090226 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:37.090378 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:43:37.090468 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:43:37.090945 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:37.086145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:35665 (local address 127.1.194.67:35493)
0808 10:43:37.086345 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:37.086362 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:37.086459 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:37.086806 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:37.086812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:37.086833 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:37.086982 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.086989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.088074 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.088079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.089747 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:37.089755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.090528 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.090531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:37.090638 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.090668 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 10:43:37.090746 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:43:37.090833 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":572}
I20260808 10:43:37.091068 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:37.086279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:35493 (local address 127.1.194.66:35665)
0808 10:43:37.086470 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:37.086477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:37.086499 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:43:37.086566 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:37.086569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:37.086680 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:37.086751 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:37.087145 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.087921 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.089857 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.090402 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.090515 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.090556 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:37.090867 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 10:43:37.090939 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 10:43:37.091347 20846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad486cb37b24cf8a3ddc476fda48d60" candidate_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" is_pre_election: true
I20260808 10:43:37.091692 20846 raft_consensus.cc:2454] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c9a5d1a1ac14a7b90c9f042437a8c35 in term 0.
I20260808 10:43:37.091657 20982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad486cb37b24cf8a3ddc476fda48d60" candidate_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" is_pre_election: true
I20260808 10:43:37.091912 20982 raft_consensus.cc:2454] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c9a5d1a1ac14a7b90c9f042437a8c35 in term 0.
I20260808 10:43:37.092250 21048 leader_election.cc:302] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c9a5d1a1ac14a7b90c9f042437a8c35, 7d132ae4b0454b238bcf88aaf1fcabfa; no voters:
I20260808 10:43:37.092571 21189 raft_consensus.cc:2781] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:43:37.092648 21189 raft_consensus.cc:482] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:43:37.092679 21189 raft_consensus.cc:3037] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:37.093829 21189 raft_consensus.cc:504] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.094223 21189 leader_election.cc:288] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [CANDIDATE]: Term 1 election: Requested vote from peers 1c15bfc18fcd419a9ba2dd46e9a2529b (127.1.194.66:35665), 7d132ae4b0454b238bcf88aaf1fcabfa (127.1.194.65:39379)
I20260808 10:43:37.094530 20982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad486cb37b24cf8a3ddc476fda48d60" candidate_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b"
I20260808 10:43:37.094667 20982 raft_consensus.cc:3037] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:37.094736 20846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad486cb37b24cf8a3ddc476fda48d60" candidate_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa"
I20260808 10:43:37.094897 20846 raft_consensus.cc:3037] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:37.095757 20982 raft_consensus.cc:2454] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c9a5d1a1ac14a7b90c9f042437a8c35 in term 1.
I20260808 10:43:37.095989 20846 raft_consensus.cc:2454] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c9a5d1a1ac14a7b90c9f042437a8c35 in term 1.
I20260808 10:43:37.096129 21046 leader_election.cc:302] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c15bfc18fcd419a9ba2dd46e9a2529b, 3c9a5d1a1ac14a7b90c9f042437a8c35; no voters:
I20260808 10:43:37.096314 21189 raft_consensus.cc:2781] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:37.096563 21189 raft_consensus.cc:686] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 1 LEADER]: Becoming Leader. State: Replica: 3c9a5d1a1ac14a7b90c9f042437a8c35, State: Running, Role: LEADER
I20260808 10:43:37.096979 21189 consensus_queue.cc:237] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } }
I20260808 10:43:37.100189 20693 catalog_manager.cc:5495] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c9a5d1a1ac14a7b90c9f042437a8c35 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c9a5d1a1ac14a7b90c9f042437a8c35" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 43741 } health_report { overall_health: HEALTHY } } }
I20260808 10:43:37.125236 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:37.121736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.1.194.67:43741)
0808 10:43:37.121899 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:37.121903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:37.121917 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:43:37.121997 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:37.121999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:37.122052 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:37.122155 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:43:37.122385 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.122931 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.123734 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:37.124021 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:37.124148 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:37.124391 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:43:37.124835 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:43:37.124926 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:37.124993 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 10:43:37.125037 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 10:43:37.130944 20982 raft_consensus.cc:1261] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 1 FOLLOWER]: Refusing update from remote peer 3c9a5d1a1ac14a7b90c9f042437a8c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:37.130945 20846 raft_consensus.cc:1261] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 1 FOLLOWER]: Refusing update from remote peer 3c9a5d1a1ac14a7b90c9f042437a8c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:37.131713 21189 consensus_queue.cc:1035] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d132ae4b0454b238bcf88aaf1fcabfa" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 39379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:37.131927 21193 consensus_queue.cc:1035] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c15bfc18fcd419a9ba2dd46e9a2529b" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 35665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:37.139246 21196 mvcc.cc:205] Tried to move back new op lower bound from 7316217106961203200 to 7316217106832429056. Current Snapshot: MvccSnapshot[applied={T|T < 7316217106961203200}]
I20260808 10:43:37.143184 21198 mvcc.cc:205] Tried to move back new op lower bound from 7316217106961203200 to 7316217106832429056. Current Snapshot: MvccSnapshot[applied={T|T < 7316217106961203200}]
I20260808 10:43:37.154244 20693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:37.154428 20693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:37.157222 20693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0e1338fb14d402390382653624aaedc: Sending DeleteTablet for 3 replicas of tablet 3ad486cb37b24cf8a3ddc476fda48d60
I20260808 10:43:37.158208 20962 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad486cb37b24cf8a3ddc476fda48d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:37 UTC) from {username='slave'} at 127.0.0.1:59804
I20260808 10:43:37.158269 21095 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad486cb37b24cf8a3ddc476fda48d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:37 UTC) from {username='slave'} at 127.0.0.1:51336
I20260808 10:43:37.158715 20824 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad486cb37b24cf8a3ddc476fda48d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:37 UTC) from {username='slave'} at 127.0.0.1:41618
I20260808 10:43:37.159194 21209 tablet_replica.cc:330] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa: stopping tablet replica
I20260808 10:43:37.159478 21207 tablet_replica.cc:330] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: stopping tablet replica
I20260808 10:43:37.159659 21209 raft_consensus.cc:2229] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:37.159842 21207 raft_consensus.cc:2229] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:37.159869 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20756
I20260808 10:43:37.160137 21207 raft_consensus.cc:2258] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:37.160147 21209 raft_consensus.cc:2258] T 3ad486cb37b24cf8a3ddc476fda48d60 P 7d132ae4b0454b238bcf88aaf1fcabfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:37.161626 21207 ts_tablet_manager.cc:1905] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:37.164628 21207 ts_tablet_manager.cc:1918] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:37.164747 21207 log.cc:1196] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/3ad486cb37b24cf8a3ddc476fda48d60
I20260808 10:43:37.165064 21207 ts_tablet_manager.cc:1939] T 3ad486cb37b24cf8a3ddc476fda48d60 P 1c15bfc18fcd419a9ba2dd46e9a2529b: Deleting consensus metadata
I20260808 10:43:37.165691 21208 tablet_replica.cc:330] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: stopping tablet replica
I20260808 10:43:37.166374 20678 catalog_manager.cc:4841] TS 1c15bfc18fcd419a9ba2dd46e9a2529b (127.1.194.66:35665): tablet 3ad486cb37b24cf8a3ddc476fda48d60 (table test-table [id=6ee765a5931a496bb48680c38932e2c4]) successfully deleted
I20260808 10:43:37.166131 21208 raft_consensus.cc:2229] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:43:37.166908 21208 raft_consensus.cc:2258] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:37.168185 21208 ts_tablet_manager.cc:1905] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:43:37.169356 20680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:39379 (error 108)
I20260808 10:43:37.170414 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:37.170112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:39379 (local address 127.0.0.1:41632)
0808 10:43:37.170256 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:37.170332 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:43:37.170459 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20896
I20260808 10:43:37.170866 21208 ts_tablet_manager.cc:1918] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:37.170954 21208 log.cc:1196] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/3ad486cb37b24cf8a3ddc476fda48d60
I20260808 10:43:37.171600 21208 ts_tablet_manager.cc:1939] T 3ad486cb37b24cf8a3ddc476fda48d60 P 3c9a5d1a1ac14a7b90c9f042437a8c35: Deleting consensus metadata
I20260808 10:43:37.172626 20678 catalog_manager.cc:4841] TS 3c9a5d1a1ac14a7b90c9f042437a8c35 (127.1.194.67:43741): tablet 3ad486cb37b24cf8a3ddc476fda48d60 (table test-table [id=6ee765a5931a496bb48680c38932e2c4]) successfully deleted
W20260808 10:43:37.174036 20680 catalog_manager.cc:4584] TS 7d132ae4b0454b238bcf88aaf1fcabfa (127.1.194.65:39379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ad486cb37b24cf8a3ddc476fda48d60: Network error: Client connection negotiation failed: client connection to 127.1.194.65:39379: connect: Connection refused (error 111)
I20260808 10:43:37.178591 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21031
I20260808 10:43:37.185005 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 20661
2026-08-08T10:43:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1107 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:37 dist-test-slave-ss4n krb5kdc[21213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:37 dist-test-slave-ss4n krb5kdc[21213](info): set up 2 sockets
Aug 08 10:43:37 dist-test-slave-ss4n krb5kdc[21213](info): commencing operation
krb5kdc: starting...
W20260808 10:43:38.319614 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:38 dist-test-slave-ss4n krb5kdc[21213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:43:38.457811 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:41557
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:41557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:38.577180 21226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:38.577468 21226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:38.577620 21226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:38.581696 21226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:38.581804 21226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:38.581849 21226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:38.581911 21226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:38.581950 21226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:38.587469 21226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:41557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:41557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:38.588585 21226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:38.589669 21226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:38.598335 21233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:38.598326 21236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:38.598331 21234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:38.598806 21226 server_base.cc:1034] running on GCE node
I20260808 10:43:38.599351 21226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:38.600499 21226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:38.601693 21226 hybrid_clock.cc:648] HybridClock initialized: now 1786185818601682 us; error 27 us; skew 500 ppm
Aug 08 10:43:38 dist-test-slave-ss4n krb5kdc[21213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:38.604605 21226 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:43:38.605528 21226 webserver.cc:460] Webserver started at http://127.1.194.126:36711/ using document root <none> and password file <none>
I20260808 10:43:38.606069 21226 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:38.606127 21226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:38.606288 21226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:38.608242 21226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "2bc752e890754afdacf2f740d2d31a51"
format_stamp: "Formatted at 2026-08-08 10:43:38 on dist-test-slave-ss4n"
server_key: "16f1a015f0c1ab5b123a2314f5b42a11"
server_key_iv: "d6985e4e860042cd02d94e2688f68381"
server_key_version: "encryptionkey@0"
I20260808 10:43:38.608799 21226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "2bc752e890754afdacf2f740d2d31a51"
format_stamp: "Formatted at 2026-08-08 10:43:38 on dist-test-slave-ss4n"
server_key: "16f1a015f0c1ab5b123a2314f5b42a11"
server_key_iv: "d6985e4e860042cd02d94e2688f68381"
server_key_version: "encryptionkey@0"
I20260808 10:43:38.612490 21226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:43:38.615041 21243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:38.616513 21226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:43:38.616681 21226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "2bc752e890754afdacf2f740d2d31a51"
format_stamp: "Formatted at 2026-08-08 10:43:38 on dist-test-slave-ss4n"
server_key: "16f1a015f0c1ab5b123a2314f5b42a11"
server_key_iv: "d6985e4e860042cd02d94e2688f68381"
server_key_version: "encryptionkey@0"
I20260808 10:43:38.616802 21226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:38.636312 21226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:38.640007 21226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:38.640251 21226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:38.648586 21226 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:41557
I20260808 10:43:38.648662 21298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:41557 every 8 connection(s)
I20260808 10:43:38.649696 21226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:38.652530 21299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:38.655508 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21226
I20260808 10:43:38.655673 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:38.656016 1801 external_mini_cluster.cc:1428] Setting key 3cdb8a3fdaeb81713810093edf9e003b
I20260808 10:43:38.658706 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: Bootstrap starting.
I20260808 10:43:38.661275 21299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:38.662145 21299 log.cc:824] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:38.664290 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: No bootstrap required, opened a new log
Aug 08 10:43:38 dist-test-slave-ss4n krb5kdc[21213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:38.669029 21299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc752e890754afdacf2f740d2d31a51" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41557 } }
I20260808 10:43:38.669276 21299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:38.669342 21299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc752e890754afdacf2f740d2d31a51, State: Initialized, Role: FOLLOWER
I20260808 10:43:38.669834 21299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc752e890754afdacf2f740d2d31a51" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41557 } }
I20260808 10:43:38.670030 21299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:38.670104 21299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:38.670200 21299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:38.671370 21299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc752e890754afdacf2f740d2d31a51" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41557 } }
I20260808 10:43:38.671734 21299 leader_election.cc:302] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bc752e890754afdacf2f740d2d31a51; no voters:
I20260808 10:43:38.672055 21299 leader_election.cc:288] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:38.672204 21305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:38.672274 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:38.657550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.1.194.126:41557)
0808 10:43:38.657988 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:38.658008 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:38.658050 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:43:38.658850 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:38.658871 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:38.659269 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:38.659613 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:38.660331 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:38.661830 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:38.663043 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:38.663712 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:38.663868 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:38.666714 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:38.666721 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:38.666737 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:38.666768 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:38.668849 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:38.669360 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:38.669367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:38.669373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:38.669539 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:38.669899 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:38.669902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:38.669904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:38.670448 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:38.670649 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:38.670940 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 10:43:38.671282 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
I20260808 10:43:38.672451 21305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [term 1 LEADER]: Becoming Leader. State: Replica: 2bc752e890754afdacf2f740d2d31a51, State: Running, Role: LEADER
I20260808 10:43:38.672751 21305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc752e890754afdacf2f740d2d31a51" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41557 } }
I20260808 10:43:38.673309 21299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:38.674125 21306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bc752e890754afdacf2f740d2d31a51. Latest consensus state: current_term: 1 leader_uuid: "2bc752e890754afdacf2f740d2d31a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc752e890754afdacf2f740d2d31a51" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41557 } } }
I20260808 10:43:38.674280 21306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:38.674540 21307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bc752e890754afdacf2f740d2d31a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc752e890754afdacf2f740d2d31a51" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41557 } } }
I20260808 10:43:38.674697 21307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51 [sys.catalog]: This master's current role is: LEADER
W20260808 10:43:38.677888 21320 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:43:38.677983 21320 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:43:38.678052 21314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:38.678721 21314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:38.684191 21314 catalog_manager.cc:1347] Generated new cluster ID: 1208e015e7e0404088e17baf0728b2ec
I20260808 10:43:38.684266 21314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:38.711416 21314 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:38.712754 21314 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:38.721185 21314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: Generated new TSK 0
I20260808 10:43:38.722059 21314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:43:38.753055 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:41557
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:38.871887 21327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:38.872151 21327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:38.872195 21327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:38.876478 21327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:38.876541 21327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:38.876600 21327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:38.882090 21327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:41557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:38.883147 21327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:38.884111 21327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:38.893944 21333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:38.893947 21335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:38.894021 21332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:38.894384 21327 server_base.cc:1034] running on GCE node
I20260808 10:43:38.894778 21327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:38.895378 21327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:38.896538 21327 hybrid_clock.cc:648] HybridClock initialized: now 1786185818896526 us; error 30 us; skew 500 ppm
Aug 08 10:43:38 dist-test-slave-ss4n krb5kdc[21213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:38.903937 21327 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:43:38.904888 21327 webserver.cc:460] Webserver started at http://127.1.194.65:44369/ using document root <none> and password file <none>
I20260808 10:43:38.905437 21327 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:38.905599 21327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:38.905809 21327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:38.907850 21327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "a17365b1840e4fb8a38cf29e252fa9ca"
format_stamp: "Formatted at 2026-08-08 10:43:38 on dist-test-slave-ss4n"
server_key: "e56aede3dd1939f56be28c3103357f7c"
server_key_iv: "f9e106592a5ab49ad7479ba2cfa4856f"
server_key_version: "encryptionkey@0"
I20260808 10:43:38.908372 21327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "a17365b1840e4fb8a38cf29e252fa9ca"
format_stamp: "Formatted at 2026-08-08 10:43:38 on dist-test-slave-ss4n"
server_key: "e56aede3dd1939f56be28c3103357f7c"
server_key_iv: "f9e106592a5ab49ad7479ba2cfa4856f"
server_key_version: "encryptionkey@0"
I20260808 10:43:38.912114 21327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 10:43:38.914665 21343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:38.916086 21327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:43:38.916258 21327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "a17365b1840e4fb8a38cf29e252fa9ca"
format_stamp: "Formatted at 2026-08-08 10:43:38 on dist-test-slave-ss4n"
server_key: "e56aede3dd1939f56be28c3103357f7c"
server_key_iv: "f9e106592a5ab49ad7479ba2cfa4856f"
server_key_version: "encryptionkey@0"
I20260808 10:43:38.916399 21327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:38.934063 21327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:38.937712 21327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:38.937935 21327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:38.938573 21327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:38.939456 21327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:38.939538 21327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:38.939613 21327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:38.939652 21327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:38.949684 21327 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:35545
I20260808 10:43:38.949720 21460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:35545 every 8 connection(s)
I20260808 10:43:38.950634 21327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:43:38 dist-test-slave-ss4n krb5kdc[21213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:38.960682 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21327
I20260808 10:43:38.960855 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:43:38.961163 1801 external_mini_cluster.cc:1428] Setting key cf40c7c9f73313df41c8a61b291f5556
I20260808 10:43:38.964888 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:38.952643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:35777 (local address 127.1.194.126:41557)
0808 10:43:38.952854 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:38.952859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:38.953668 (+ 809us) server_negotiation.cc:407] Connection header received
0808 10:43:38.954988 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:38.954991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:38.955037 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:38.955117 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:38.956386 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:38.956924 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:38.958192 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:38.958422 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:38.958508 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:38.961138 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:38.961142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:38.961145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:38.961182 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:38.962846 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:38.963363 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:38.963368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:38.963370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:38.963458 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:38.963812 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:38.963816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:38.963817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:38.963999 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:38.964108 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:38.964598 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 10:43:38.964729 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 10:43:38.965770 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:38.952980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:41557 (local address 127.1.194.65:35777)
0808 10:43:38.953445 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:38.953540 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:38.954774 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:38.955290 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:38.955297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:38.955715 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:38.956210 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:38.956224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:38.957048 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:38.957051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:38.958066 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:38.958076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:38.958560 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:38.958563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:38.958745 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:38.959337 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:38.959353 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:38.960971 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:38.962971 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:38.962977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:38.962991 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:38.963252 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:38.963554 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:38.963558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:38.963560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:38.963710 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:38.964118 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:38.964126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:38.964365 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 10:43:38.964557 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 10:43:38.964885 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260808 10:43:38.967103 21461 heartbeater.cc:347] Connected to a master server at 127.1.194.126:41557
I20260808 10:43:38.967371 21461 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:38.968007 21461 heartbeater.cc:511] Master 127.1.194.126:41557 requested a full tablet report, sending...
I20260808 10:43:38.969962 21260 ts_manager.cc:194] Registered new tserver with Master: a17365b1840e4fb8a38cf29e252fa9ca (127.1.194.65:35545)
I20260808 10:43:38.971720 21260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:35777
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:43:38.993594 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:41557
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:39.104795 21466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:39.105105 21466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:39.105182 21466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:39.109387 21466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:39.109474 21466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:39.109586 21466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:39.115037 21466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:41557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:39.116114 21466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:39.117146 21466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:39.126964 21474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:39.127019 21471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:39.126964 21472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:39.127172 21466 server_base.cc:1034] running on GCE node
I20260808 10:43:39.127676 21466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:39.128289 21466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:39.129447 21466 hybrid_clock.cc:648] HybridClock initialized: now 1786185819129416 us; error 50 us; skew 500 ppm
Aug 08 10:43:39 dist-test-slave-ss4n krb5kdc[21213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:39.132409 21466 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:43:39.133345 21466 webserver.cc:460] Webserver started at http://127.1.194.66:40665/ using document root <none> and password file <none>
I20260808 10:43:39.133919 21466 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:39.133981 21466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:39.134200 21466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:39.136173 21466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "5009da9212604c7b84e99f38b0bc84ab"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "452981a15d84ad92f889ff8e1e257c07"
server_key_iv: "0f9fe518d7ba43a6a5377975a97c9f11"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.136723 21466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "5009da9212604c7b84e99f38b0bc84ab"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "452981a15d84ad92f889ff8e1e257c07"
server_key_iv: "0f9fe518d7ba43a6a5377975a97c9f11"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.140388 21466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:43:39.142833 21483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.144047 21466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:43:39.144177 21466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "5009da9212604c7b84e99f38b0bc84ab"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "452981a15d84ad92f889ff8e1e257c07"
server_key_iv: "0f9fe518d7ba43a6a5377975a97c9f11"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.144282 21466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:39.157598 21466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:39.160348 21466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:39.160549 21466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:39.161128 21466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:39.161947 21466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:39.161993 21466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.162065 21466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:39.162103 21466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.170881 21466 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:39251
I20260808 10:43:39.170912 21599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:39251 every 8 connection(s)
I20260808 10:43:39.171725 21466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:39.181257 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21466
Aug 08 10:43:39 dist-test-slave-ss4n krb5kdc[21213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:39.181447 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:43:39.181828 1801 external_mini_cluster.cc:1428] Setting key 6f03ab8b77ae87b8d2a3d5a4340f562d
I20260808 10:43:39.186120 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.173575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:51609 (local address 127.1.194.126:41557)
0808 10:43:39.173714 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.173718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.174628 (+ 910us) server_negotiation.cc:407] Connection header received
0808 10:43:39.176001 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.176004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.176073 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.176182 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:39.177318 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.177961 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.179117 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.179350 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.179470 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.182106 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:39.182111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:39.182115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:39.182149 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:39.183940 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:39.184504 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:39.184508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:39.184510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:39.184599 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:39.184943 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:39.184947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:39.184949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:39.185141 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:39.185256 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.185820 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.185941 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:43:39.186918 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.173869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:41557 (local address 127.1.194.66:51609)
0808 10:43:39.174408 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.174449 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.175788 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.176347 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.176353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.176702 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:39.177162 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.177175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.178120 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.178123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.178981 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.178990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.179510 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.179515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.179689 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.180335 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:39.180352 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:39.181879 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:39.184112 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:39.184123 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:39.184136 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:39.184385 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:39.184696 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:39.184700 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:39.184702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:39.184843 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:39.185254 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:39.185261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:39.185569 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.185762 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.186092 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":152,"threads_started":1}
I20260808 10:43:39.188126 21600 heartbeater.cc:347] Connected to a master server at 127.1.194.126:41557
I20260808 10:43:39.188436 21600 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:39.189131 21600 heartbeater.cc:511] Master 127.1.194.126:41557 requested a full tablet report, sending...
I20260808 10:43:39.190435 21260 ts_manager.cc:194] Registered new tserver with Master: 5009da9212604c7b84e99f38b0bc84ab (127.1.194.66:39251)
I20260808 10:43:39.191435 21260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:51609
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:43:39.213358 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:41557
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:39.325088 21605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:39.325348 21605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:39.325394 21605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:39.329667 21605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:39.329731 21605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:39.329792 21605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:39.335206 21605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:41557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:39.336261 21605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:39.337251 21605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:39.347441 21611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:39.347494 21610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:39.347478 21613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:39.347460 21605 server_base.cc:1034] running on GCE node
I20260808 10:43:39.348212 21605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:39.348883 21605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:39.350107 21605 hybrid_clock.cc:648] HybridClock initialized: now 1786185819350095 us; error 28 us; skew 500 ppm
Aug 08 10:43:39 dist-test-slave-ss4n krb5kdc[21213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:39.352926 21605 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:43:39.353945 21605 webserver.cc:460] Webserver started at http://127.1.194.67:46115/ using document root <none> and password file <none>
I20260808 10:43:39.354473 21605 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:39.354537 21605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:39.354704 21605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:39.356694 21605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "634f37fa6bd04ede9e27518367aa9cdc"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "aed36ea7980b33ab409deaee9c954d44"
server_key_iv: "336d6fd10ad65549938f7f1f6171f77f"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.357210 21605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "634f37fa6bd04ede9e27518367aa9cdc"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "aed36ea7980b33ab409deaee9c954d44"
server_key_iv: "336d6fd10ad65549938f7f1f6171f77f"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.360914 21605 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:43:39.363413 21620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.364822 21605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:39.364920 21605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "634f37fa6bd04ede9e27518367aa9cdc"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "aed36ea7980b33ab409deaee9c954d44"
server_key_iv: "336d6fd10ad65549938f7f1f6171f77f"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.365006 21605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:39.387518 21605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:39.390807 21605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:39.390995 21605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:39.391590 21605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:39.392462 21605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:39.392516 21605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.392563 21605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:39.392589 21605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.402697 21605 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:44281
I20260808 10:43:39.402717 21738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:44281 every 8 connection(s)
I20260808 10:43:39.403656 21605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:39.411036 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21605
I20260808 10:43:39.411180 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:43:39.411509 1801 external_mini_cluster.cc:1428] Setting key 84f9448db22119816ab7c0c4b6bf676e
Aug 08 10:43:39 dist-test-slave-ss4n krb5kdc[21213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:39.418296 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.405712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:44285 (local address 127.1.194.126:41557)
0808 10:43:39.405859 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.405863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.406738 (+ 875us) server_negotiation.cc:407] Connection header received
0808 10:43:39.408031 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.408033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.408077 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.408157 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:39.409297 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.409834 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.411006 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.411362 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.411477 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.414643 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:39.414645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:39.414647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:39.414670 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:39.416167 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:39.416755 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:39.416757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:39.416758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:39.416838 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:39.417280 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:39.417282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:39.417283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:39.417457 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:39.417604 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.418076 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.418176 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:43:39.419193 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.406092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:41557 (local address 127.1.194.67:44285)
0808 10:43:39.406577 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.406630 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.407843 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.408337 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.408344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.408702 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:39.409165 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.409178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.409942 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.409945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.410861 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.410870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.411510 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.411513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.411737 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.412447 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:39.412474 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:39.414475 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:39.416290 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:39.416295 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:39.416308 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:39.416654 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:39.416987 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:39.416991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:39.416992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:39.417186 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:39.417598 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:39.417604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:39.417860 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.418024 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.418321 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":145,"threads_started":1}
I20260808 10:43:39.420439 21739 heartbeater.cc:347] Connected to a master server at 127.1.194.126:41557
I20260808 10:43:39.420686 21739 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:39.421298 21739 heartbeater.cc:511] Master 127.1.194.126:41557 requested a full tablet report, sending...
I20260808 10:43:39.422390 21260 ts_manager.cc:194] Registered new tserver with Master: 634f37fa6bd04ede9e27518367aa9cdc (127.1.194.67:44281)
I20260808 10:43:39.422984 21260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:44285
I20260808 10:43:39.425890 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:43:39.435917 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.427894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.1.194.126:41557)
0808 10:43:39.428051 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.428057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.428252 (+ 195us) server_negotiation.cc:407] Connection header received
0808 10:43:39.428400 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.428403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.428446 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.428533 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:39.429267 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.429876 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.430827 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.431059 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.431182 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.432786 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:39.432790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:39.432792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:39.432816 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:39.434061 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:39.434572 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:39.434576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:39.434578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:39.434656 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:39.435001 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:39.435005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:39.435006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:39.435221 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:39.435385 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.435663 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.435754 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:43:39.438899 21260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:43:39.441946 21260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:43:39.457297 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.451702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59422 (local address 127.1.194.65:35545)
0808 10:43:39.452074 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.452093 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.452405 (+ 312us) server_negotiation.cc:407] Connection header received
0808 10:43:39.452506 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.452513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.452698 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.452878 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.453122 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.454218 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.455658 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.456504 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.456623 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.456772 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.457015 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.457127 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":71,"threads_started":1}
I20260808 10:43:39.457358 21753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.450904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:35545 (local address 127.0.0.1:59422)
0808 10:43:39.452278 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.452292 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.452394 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.452872 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.452876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.452895 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.453013 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.453023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.454371 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.454374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.455513 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.455520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.456650 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.456655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.456787 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.456832 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.456969 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.457147 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"mutex_wait_us":44,"thread_start_us":119,"threads_started":1}
I20260808 10:43:39.457417 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.451752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.1.194.67:44281)
0808 10:43:39.452246 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.452261 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.452763 (+ 502us) server_negotiation.cc:407] Connection header received
0808 10:43:39.452841 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.452846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.453026 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.453164 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.453578 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.454624 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.456055 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.456681 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.456796 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.456950 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.457133 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.457238 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":161,"threads_started":1}
I20260808 10:43:39.457605 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.451580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44281 (local address 127.0.0.1:56982)
0808 10:43:39.452570 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.452583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.452664 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.453316 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.453319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.453337 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.453414 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.453419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.454829 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.454835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.455898 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.455916 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.456858 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.456861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.456969 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.456994 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.457082 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.457164 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":7,"thread_start_us":99,"threads_started":1}
I20260808 10:43:39.457963 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.450858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:39251 (local address 127.0.0.1:43618)
0808 10:43:39.451497 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.451548 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.451687 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.452677 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.452685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.452711 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.452847 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.452854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.454099 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.454105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.455503 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.455514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.457297 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.457300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.457401 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.457432 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.457750 (+ 318us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.457846 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260808 10:43:39.458267 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.450982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.1.194.66:39251)
0808 10:43:39.451367 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.451381 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.451851 (+ 470us) server_negotiation.cc:407] Connection header received
0808 10:43:39.452100 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.452107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.452327 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.452479 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.452973 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.453965 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.455658 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.456504 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.456623 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.456772 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.458003 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.458110 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":144,"threads_started":1}
I20260808 10:43:39.460681 21534 tablet_service.cc:1467] Processing CreateTablet for tablet 350e2ca4315d4f2a85f1b71863c99303 (DEFAULT_TABLE table=test-table [id=2bd9ddcec37843aeb9d1cff859bbd475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:39.460716 21672 tablet_service.cc:1467] Processing CreateTablet for tablet 350e2ca4315d4f2a85f1b71863c99303 (DEFAULT_TABLE table=test-table [id=2bd9ddcec37843aeb9d1cff859bbd475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:39.460716 21393 tablet_service.cc:1467] Processing CreateTablet for tablet 350e2ca4315d4f2a85f1b71863c99303 (DEFAULT_TABLE table=test-table [id=2bd9ddcec37843aeb9d1cff859bbd475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:39.461740 21534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 350e2ca4315d4f2a85f1b71863c99303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:39.461781 21672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 350e2ca4315d4f2a85f1b71863c99303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:39.461774 21393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 350e2ca4315d4f2a85f1b71863c99303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:39.468145 21757 tablet_bootstrap.cc:492] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Bootstrap starting.
I20260808 10:43:39.469851 21759 tablet_bootstrap.cc:492] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: Bootstrap starting.
I20260808 10:43:39.470427 21757 tablet_bootstrap.cc:654] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:39.471453 21757 log.cc:824] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:39.471648 21759 tablet_bootstrap.cc:654] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:39.472129 21758 tablet_bootstrap.cc:492] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Bootstrap starting.
I20260808 10:43:39.472514 21759 log.cc:824] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:39.473834 21757 tablet_bootstrap.cc:492] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: No bootstrap required, opened a new log
I20260808 10:43:39.474051 21757 ts_tablet_manager.cc:1397] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:43:39.474395 21758 tablet_bootstrap.cc:654] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:39.474502 21759 tablet_bootstrap.cc:492] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: No bootstrap required, opened a new log
I20260808 10:43:39.474686 21759 ts_tablet_manager.cc:1397] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:39.475428 21758 log.cc:824] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:39.477011 21758 tablet_bootstrap.cc:492] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: No bootstrap required, opened a new log
I20260808 10:43:39.477216 21758 ts_tablet_manager.cc:1397] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:39.478451 21757 raft_consensus.cc:348] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.478757 21757 raft_consensus.cc:374] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:39.478833 21757 raft_consensus.cc:729] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5009da9212604c7b84e99f38b0bc84ab, State: Initialized, Role: FOLLOWER
I20260808 10:43:39.479315 21757 consensus_queue.cc:260] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.479251 21759 raft_consensus.cc:348] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.479498 21759 raft_consensus.cc:374] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:39.479561 21759 raft_consensus.cc:729] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17365b1840e4fb8a38cf29e252fa9ca, State: Initialized, Role: FOLLOWER
I20260808 10:43:39.480051 21600 heartbeater.cc:503] Master 127.1.194.126:41557 was elected leader, sending a full tablet report...
I20260808 10:43:39.480022 21759 consensus_queue.cc:260] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.480393 21757 ts_tablet_manager.cc:1428] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:39.480865 21759 ts_tablet_manager.cc:1428] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:39.481243 21461 heartbeater.cc:503] Master 127.1.194.126:41557 was elected leader, sending a full tablet report...
I20260808 10:43:39.481575 21758 raft_consensus.cc:348] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.481830 21758 raft_consensus.cc:374] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:39.481884 21758 raft_consensus.cc:729] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634f37fa6bd04ede9e27518367aa9cdc, State: Initialized, Role: FOLLOWER
I20260808 10:43:39.482308 21758 consensus_queue.cc:260] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.483047 21739 heartbeater.cc:503] Master 127.1.194.126:41557 was elected leader, sending a full tablet report...
I20260808 10:43:39.483258 21758 ts_tablet_manager.cc:1428] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:43:39.545699 21763 raft_consensus.cc:482] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:39.545957 21763 raft_consensus.cc:504] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.547346 21763 leader_election.cc:288] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a17365b1840e4fb8a38cf29e252fa9ca (127.1.194.65:35545), 634f37fa6bd04ede9e27518367aa9cdc (127.1.194.67:44281)
I20260808 10:43:39.551542 21766 raft_consensus.cc:482] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:39.551734 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.547567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:35545 (local address 127.1.194.66:45031)
0808 10:43:39.547898 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.547914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.548001 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.548421 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.548428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.548455 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.548599 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.548606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.549417 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.549421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.550519 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.550530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.551242 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.551249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.551389 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.551413 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.551497 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.551584 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":4}
I20260808 10:43:39.551753 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.548062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:45031 (local address 127.1.194.65:35545)
0808 10:43:39.548199 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.548205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.548219 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:43:39.548269 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.548271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.548310 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.548407 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.548772 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.549308 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.550701 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.551114 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.551210 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.551250 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.551527 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.551630 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:43:39.551736 21766 raft_consensus.cc:504] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.551990 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.547618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:44281 (local address 127.1.194.66:56699)
0808 10:43:39.548154 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.548169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.548314 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.548607 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.548610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.548626 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.548751 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.548757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.549724 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.549727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.550564 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.550575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.551345 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.551349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.551460 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.551493 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.551764 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.551842 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":195,"threads_started":1}
I20260808 10:43:39.552215 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.547751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:56699 (local address 127.1.194.67:44281)
0808 10:43:39.547912 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.547919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.548274 (+ 355us) server_negotiation.cc:407] Connection header received
0808 10:43:39.548451 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.548453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.548505 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.548571 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.548880 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.549597 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.550732 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.551205 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.551357 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.551408 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.551618 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.551693 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:43:39.552418 21414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350e2ca4315d4f2a85f1b71863c99303" candidate_uuid: "5009da9212604c7b84e99f38b0bc84ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" is_pre_election: true
I20260808 10:43:39.552734 21414 raft_consensus.cc:2454] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5009da9212604c7b84e99f38b0bc84ab in term 0.
I20260808 10:43:39.552999 21692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350e2ca4315d4f2a85f1b71863c99303" candidate_uuid: "5009da9212604c7b84e99f38b0bc84ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634f37fa6bd04ede9e27518367aa9cdc" is_pre_election: true
I20260808 10:43:39.553232 21692 raft_consensus.cc:2454] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5009da9212604c7b84e99f38b0bc84ab in term 0.
I20260808 10:43:39.553310 21485 leader_election.cc:302] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5009da9212604c7b84e99f38b0bc84ab, a17365b1840e4fb8a38cf29e252fa9ca; no voters:
I20260808 10:43:39.554093 21766 leader_election.cc:288] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5009da9212604c7b84e99f38b0bc84ab (127.1.194.66:39251), a17365b1840e4fb8a38cf29e252fa9ca (127.1.194.65:35545)
I20260808 10:43:39.554301 21763 raft_consensus.cc:2781] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:43:39.554392 21763 raft_consensus.cc:482] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:43:39.554454 21763 raft_consensus.cc:3037] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:39.555805 21763 raft_consensus.cc:504] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.556480 21692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350e2ca4315d4f2a85f1b71863c99303" candidate_uuid: "5009da9212604c7b84e99f38b0bc84ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634f37fa6bd04ede9e27518367aa9cdc"
I20260808 10:43:39.556480 21414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350e2ca4315d4f2a85f1b71863c99303" candidate_uuid: "5009da9212604c7b84e99f38b0bc84ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17365b1840e4fb8a38cf29e252fa9ca"
I20260808 10:43:39.556610 21692 raft_consensus.cc:3037] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:39.556636 21414 raft_consensus.cc:3037] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:39.557878 21414 raft_consensus.cc:2454] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5009da9212604c7b84e99f38b0bc84ab in term 1.
I20260808 10:43:39.557878 21692 raft_consensus.cc:2454] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5009da9212604c7b84e99f38b0bc84ab in term 1.
I20260808 10:43:39.558025 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.553282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:39251 (local address 127.1.194.67:42481)
0808 10:43:39.553772 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.553790 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.553878 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.554219 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.554223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.554256 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.554366 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.554375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.555352 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.555357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.556556 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.556567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.557608 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.557612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.557721 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.557743 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.557816 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.557889 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":47}
I20260808 10:43:39.558292 21484 leader_election.cc:302] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5009da9212604c7b84e99f38b0bc84ab, 634f37fa6bd04ede9e27518367aa9cdc; no voters:
I20260808 10:43:39.558414 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.553870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:42481 (local address 127.1.194.66:39251)
0808 10:43:39.553986 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.553996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.554007 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:43:39.554039 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.554041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.554106 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.554181 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.554507 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.555235 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.556711 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.557350 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.557474 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.557547 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.558167 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.558251 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:43:39.558660 21769 raft_consensus.cc:2781] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:39.558772 21763 leader_election.cc:288] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [CANDIDATE]: Term 1 election: Requested vote from peers a17365b1840e4fb8a38cf29e252fa9ca (127.1.194.65:35545), 634f37fa6bd04ede9e27518367aa9cdc (127.1.194.67:44281)
I20260808 10:43:39.558822 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.554192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:33593 (local address 127.1.194.65:35545)
0808 10:43:39.554388 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.554392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.554453 (+ 61us) server_negotiation.cc:407] Connection header received
0808 10:43:39.554620 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.554623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.554675 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.554756 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:39.555056 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.555742 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.557388 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.558051 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.558153 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.558201 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.558611 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.558697 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260808 10:43:39.558911 21769 raft_consensus.cc:686] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 1 LEADER]: Becoming Leader. State: Replica: 5009da9212604c7b84e99f38b0bc84ab, State: Running, Role: LEADER
I20260808 10:43:39.558951 21554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350e2ca4315d4f2a85f1b71863c99303" candidate_uuid: "634f37fa6bd04ede9e27518367aa9cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5009da9212604c7b84e99f38b0bc84ab" is_pre_election: true
I20260808 10:43:39.559235 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.553740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:35545 (local address 127.1.194.67:33593)
0808 10:43:39.554356 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.554370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:39.554519 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:39.554797 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:39.554800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:39.554819 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:39.554931 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.554937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.555873 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.555877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.557234 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:39.557244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.558323 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.558327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:39.558439 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.558472 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:43:39.559019 (+ 547us) client_negotiation.cc:240] Negotiation successful
0808 10:43:39.559097 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":20,"thread_start_us":165,"threads_started":1}
I20260808 10:43:39.559284 21769 consensus_queue.cc:237] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } }
I20260808 10:43:39.559656 21414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350e2ca4315d4f2a85f1b71863c99303" candidate_uuid: "634f37fa6bd04ede9e27518367aa9cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" is_pre_election: true
I20260808 10:43:39.559841 21414 raft_consensus.cc:2379] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 634f37fa6bd04ede9e27518367aa9cdc in current term 1: Already voted for candidate 5009da9212604c7b84e99f38b0bc84ab in this term.
I20260808 10:43:39.560310 21622 leader_election.cc:302] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 634f37fa6bd04ede9e27518367aa9cdc; no voters: 5009da9212604c7b84e99f38b0bc84ab, a17365b1840e4fb8a38cf29e252fa9ca
I20260808 10:43:39.560617 21766 raft_consensus.cc:2726] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 10:43:39.562989 21260 catalog_manager.cc:5495] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 5009da9212604c7b84e99f38b0bc84ab (127.1.194.66). New cstate: current_term: 1 leader_uuid: "5009da9212604c7b84e99f38b0bc84ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5009da9212604c7b84e99f38b0bc84ab" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260808 10:43:39.579650 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.576360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.1.194.66:39251)
0808 10:43:39.576485 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.576488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.576501 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:43:39.576625 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.576627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.576678 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.576764 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:43:39.576985 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.577461 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.578381 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.578602 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.578678 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.578933 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:43:39.579304 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:43:39.579409 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.579491 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.579518 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:43:39.585098 21414 raft_consensus.cc:1261] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 1 FOLLOWER]: Refusing update from remote peer 5009da9212604c7b84e99f38b0bc84ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:39.585140 21692 raft_consensus.cc:1261] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 1 FOLLOWER]: Refusing update from remote peer 5009da9212604c7b84e99f38b0bc84ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:39.585825 21769 consensus_queue.cc:1035] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17365b1840e4fb8a38cf29e252fa9ca" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:39.586032 21770 consensus_queue.cc:1035] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "634f37fa6bd04ede9e27518367aa9cdc" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:39.596016 21774 mvcc.cc:205] Tried to move back new op lower bound from 7316217117014020096 to 7316217116918243328. Current Snapshot: MvccSnapshot[applied={T|T < 7316217117014020096 or (T in {7316217117014020096})}]
I20260808 10:43:39.602206 21260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:39.602398 21260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:39.604874 21260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bc752e890754afdacf2f740d2d31a51: Sending DeleteTablet for 3 replicas of tablet 350e2ca4315d4f2a85f1b71863c99303
I20260808 10:43:39.605712 21393 tablet_service.cc:1514] Processing DeleteTablet for tablet 350e2ca4315d4f2a85f1b71863c99303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:39 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:59422
I20260808 10:43:39.605768 21534 tablet_service.cc:1514] Processing DeleteTablet for tablet 350e2ca4315d4f2a85f1b71863c99303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:39 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:43618
I20260808 10:43:39.606248 21783 tablet_replica.cc:330] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca: stopping tablet replica
I20260808 10:43:39.606266 21672 tablet_service.cc:1514] Processing DeleteTablet for tablet 350e2ca4315d4f2a85f1b71863c99303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:39 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:56982
I20260808 10:43:39.606731 21784 tablet_replica.cc:330] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: stopping tablet replica
I20260808 10:43:39.606726 21783 raft_consensus.cc:2229] T 350e2ca4315d4f2a85f1b71863c99303 P a17365b1840e4fb8a38cf29e252fa9ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:39.607007 21784 raft_consensus.cc:2229] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:43:39.607112 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21327
I20260808 10:43:39.607750 21784 raft_consensus.cc:2258] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:39.610318 21784 ts_tablet_manager.cc:1905] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:39.613181 21784 ts_tablet_manager.cc:1918] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:39.613294 21784 log.cc:1196] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/350e2ca4315d4f2a85f1b71863c99303
I20260808 10:43:39.613636 21785 tablet_replica.cc:330] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: stopping tablet replica
I20260808 10:43:39.613687 21784 ts_tablet_manager.cc:1939] T 350e2ca4315d4f2a85f1b71863c99303 P 5009da9212604c7b84e99f38b0bc84ab: Deleting consensus metadata
I20260808 10:43:39.614177 21785 raft_consensus.cc:2229] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:39.614569 21785 raft_consensus.cc:2258] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:39.614934 21244 catalog_manager.cc:4841] TS 5009da9212604c7b84e99f38b0bc84ab (127.1.194.66:39251): tablet 350e2ca4315d4f2a85f1b71863c99303 (table test-table [id=2bd9ddcec37843aeb9d1cff859bbd475]) successfully deleted
W20260808 10:43:39.615988 21245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:35545 (error 108)
I20260808 10:43:39.616252 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21466
I20260808 10:43:39.618296 21785 ts_tablet_manager.cc:1905] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:39.620270 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.619920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:35545 (local address 127.0.0.1:59436)
0808 10:43:39.620091 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:39.620170 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 10:43:39.620466 21245 catalog_manager.cc:4584] TS a17365b1840e4fb8a38cf29e252fa9ca (127.1.194.65:35545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 350e2ca4315d4f2a85f1b71863c99303: Network error: Client connection negotiation failed: client connection to 127.1.194.65:35545: connect: Connection refused (error 111)
I20260808 10:43:39.621120 21785 ts_tablet_manager.cc:1918] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:39.621245 21785 log.cc:1196] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/350e2ca4315d4f2a85f1b71863c99303
I20260808 10:43:39.621943 21785 ts_tablet_manager.cc:1939] T 350e2ca4315d4f2a85f1b71863c99303 P 634f37fa6bd04ede9e27518367aa9cdc: Deleting consensus metadata
I20260808 10:43:39.623111 21244 catalog_manager.cc:4841] TS 634f37fa6bd04ede9e27518367aa9cdc (127.1.194.67:44281): tablet 350e2ca4315d4f2a85f1b71863c99303 (table test-table [id=2bd9ddcec37843aeb9d1cff859bbd475]) successfully deleted
I20260808 10:43:39.624646 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21605
I20260808 10:43:39.633877 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21226
2026-08-08T10:43:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2449 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T10:43:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:39Z Disabled control of system clock
I20260808 10:43:39.667507 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42913
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:43:39.781868 21791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:39.782148 21791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:39.782204 21791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:39.786180 21791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:39.786233 21791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:39.786249 21791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:39.786268 21791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:39.786283 21791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:39.791031 21791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42913
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:39.792013 21791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:39.793289 21791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:39.801822 21799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:39.801857 21797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:39.801957 21796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:39.802004 21791 server_base.cc:1034] running on GCE node
I20260808 10:43:39.802794 21791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:39.804013 21791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:39.805248 21791 hybrid_clock.cc:648] HybridClock initialized: now 1786185819805219 us; error 48 us; skew 500 ppm
I20260808 10:43:39.807164 21791 webserver.cc:460] Webserver started at http://127.1.194.126:33053/ using document root <none> and password file <none>
I20260808 10:43:39.807729 21791 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:39.807830 21791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:39.808059 21791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:39.810153 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "5c00ee9750b543ff83a2fd5b4f83a808"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "11f107a9d1d563cf106b7c9334dce4af"
server_key_iv: "7e15474f478bce03a4d53d2d6f29fa57"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.810730 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "5c00ee9750b543ff83a2fd5b4f83a808"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "11f107a9d1d563cf106b7c9334dce4af"
server_key_iv: "7e15474f478bce03a4d53d2d6f29fa57"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.814275 21791 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:43:39.816615 21805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:39.817979 21791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:39.818141 21791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "5c00ee9750b543ff83a2fd5b4f83a808"
format_stamp: "Formatted at 2026-08-08 10:43:39 on dist-test-slave-ss4n"
server_key: "11f107a9d1d563cf106b7c9334dce4af"
server_key_iv: "7e15474f478bce03a4d53d2d6f29fa57"
server_key_version: "encryptionkey@0"
I20260808 10:43:39.818255 21791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:39.832078 21791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:39.832749 21791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:39.832952 21791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:39.839994 21791 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:42913
I20260808 10:43:39.840000 21858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:42913 every 8 connection(s)
I20260808 10:43:39.840930 21791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:39.843827 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21791
I20260808 10:43:39.843966 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:39.844148 21859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:39.844317 1801 external_mini_cluster.cc:1428] Setting key 3bdb2d83fbff49e53a4156b91ef6ce85
I20260808 10:43:39.850543 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808: Bootstrap starting.
I20260808 10:43:39.852744 21859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:39.853636 21859 log.cc:824] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:39.854113 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:39.845751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.1.194.126:42913)
0808 10:43:39.846373 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:39.846388 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:39.846423 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:43:39.847388 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:39.847408 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:39.847791 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:39.848142 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:39.848569 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.849579 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.850495 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:39.851094 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:39.851272 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:39.852224 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:39.852229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:39.852241 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:39.852616 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:39.852670 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:39.852689 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:39.852797 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:39.853000 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 10:43:39.853258 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260808 10:43:39.855680 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808: No bootstrap required, opened a new log
I20260808 10:43:39.859409 21859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42913 } }
I20260808 10:43:39.859629 21859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:39.859663 21859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c00ee9750b543ff83a2fd5b4f83a808, State: Initialized, Role: FOLLOWER
I20260808 10:43:39.860138 21859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42913 } }
I20260808 10:43:39.860329 21859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:39.860405 21859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:39.860508 21859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:39.861392 21859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42913 } }
I20260808 10:43:39.861752 21859 leader_election.cc:302] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c00ee9750b543ff83a2fd5b4f83a808; no voters:
I20260808 10:43:39.862064 21859 leader_election.cc:288] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:39.862203 21864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:39.862455 21864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [term 1 LEADER]: Becoming Leader. State: Replica: 5c00ee9750b543ff83a2fd5b4f83a808, State: Running, Role: LEADER
I20260808 10:43:39.862797 21864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42913 } }
I20260808 10:43:39.863356 21859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:39.864589 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42913 } } }
I20260808 10:43:39.864635 21866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c00ee9750b543ff83a2fd5b4f83a808. Latest consensus state: current_term: 1 leader_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c00ee9750b543ff83a2fd5b4f83a808" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42913 } } }
I20260808 10:43:39.864719 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:39.864733 21866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:39.865118 21875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:39.867796 21875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:39.872695 21875 catalog_manager.cc:1347] Generated new cluster ID: fa81e2b6288a4be7ad81ae8d4cb467dd
I20260808 10:43:39.872761 21875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:39.878842 21875 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:39.879698 21875 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:39.890105 21875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808: Generated new TSK 0
I20260808 10:43:39.890799 21875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:39.898157 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42913
--builtin_ntp_servers=127.1.194.84:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:43:40.013288 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:40.013620 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:40.013695 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:40.017768 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:40.017863 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:40.017956 21885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:40.022917 21885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:40.023994 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:40.025394 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:43:40.035009 21885 server_base.cc:1034] running on GCE node
W20260808 10:43:40.034955 21891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:40.034970 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:40.034970 21890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:40.035576 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:40.036223 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:40.037416 21885 hybrid_clock.cc:648] HybridClock initialized: now 1786185820037410 us; error 44 us; skew 500 ppm
I20260808 10:43:40.039314 21885 webserver.cc:460] Webserver started at http://127.1.194.65:45551/ using document root <none> and password file <none>
I20260808 10:43:40.039889 21885 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:40.039984 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:40.040223 21885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:40.042415 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "a0e2e65eff4b456c906a7337e8feeb44"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "ad0a8a8a1a21ac0e4bfeda9c611c90bf"
server_key_iv: "46c08fe6f261c45217f10aff0db3dc49"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.043004 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "a0e2e65eff4b456c906a7337e8feeb44"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "ad0a8a8a1a21ac0e4bfeda9c611c90bf"
server_key_iv: "46c08fe6f261c45217f10aff0db3dc49"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.047068 21885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:43:40.049991 21900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.051388 21885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:40.051559 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "a0e2e65eff4b456c906a7337e8feeb44"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "ad0a8a8a1a21ac0e4bfeda9c611c90bf"
server_key_iv: "46c08fe6f261c45217f10aff0db3dc49"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.051717 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:40.067394 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:40.068188 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:40.068425 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:40.069093 21885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:40.070209 21885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:40.070307 21885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.070389 21885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:40.070448 21885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.080098 21885 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:40549
I20260808 10:43:40.080129 22017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:40549 every 8 connection(s)
I20260808 10:43:40.081126 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:43:40.085549 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 21885
I20260808 10:43:40.085709 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:43:40.086009 1801 external_mini_cluster.cc:1428] Setting key 8720a0a0300b862461d4f0b64b36ba95
I20260808 10:43:40.088215 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42913
--builtin_ntp_servers=127.1.194.84:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:43:40.091455 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.083144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:43237 (local address 127.1.194.126:42913)
0808 10:43:40.083304 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.083308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.084476 (+ 1168us) server_negotiation.cc:407] Connection header received
0808 10:43:40.086251 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.086255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.086318 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.086435 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:40.087485 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.088029 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.089325 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.089630 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.089746 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.090928 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:40.090931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:40.090934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:40.090948 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:40.090971 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:40.090985 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:40.091064 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.091242 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.091311 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:43:40.092356 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.083575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42913 (local address 127.1.194.65:43237)
0808 10:43:40.084289 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.084347 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.085947 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.086630 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.086639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.086751 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:40.087355 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.087377 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.088174 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.088178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.089190 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.089200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.089792 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.089796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.089971 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.090652 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:40.090654 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:40.090696 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:40.090763 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:40.091078 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:40.091090 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:40.091098 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.091242 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.091543 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260808 10:43:40.093654 22018 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42913
I20260808 10:43:40.093931 22018 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:40.094480 22018 heartbeater.cc:511] Master 127.1.194.126:42913 requested a full tablet report, sending...
I20260808 10:43:40.095986 21822 ts_manager.cc:194] Registered new tserver with Master: a0e2e65eff4b456c906a7337e8feeb44 (127.1.194.65:40549)
I20260808 10:43:40.097127 21822 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.65:43237
W20260808 10:43:40.201269 22021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:40.201627 22021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:40.201705 22021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:40.205801 22021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:40.205871 22021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:40.205958 22021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:40.210716 22021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:40.211777 22021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:40.213119 22021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:40.222733 22027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:40.222805 22028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:40.222828 22021 server_base.cc:1034] running on GCE node
W20260808 10:43:40.222733 22030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:40.223454 22021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:40.224143 22021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:40.225350 22021 hybrid_clock.cc:648] HybridClock initialized: now 1786185820225297 us; error 75 us; skew 500 ppm
I20260808 10:43:40.227269 22021 webserver.cc:460] Webserver started at http://127.1.194.66:38113/ using document root <none> and password file <none>
I20260808 10:43:40.227849 22021 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:40.227944 22021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:40.228155 22021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:40.230274 22021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "6e2732e443424195b9090081c5f70700"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "61aca29042d5a86a7469b124f6f942d9"
server_key_iv: "f2df4dc09175ae5f9d418338f8a4d38a"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.230875 22021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "6e2732e443424195b9090081c5f70700"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "61aca29042d5a86a7469b124f6f942d9"
server_key_iv: "f2df4dc09175ae5f9d418338f8a4d38a"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.234771 22021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:43:40.237110 22037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.238409 22021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:40.238586 22021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "6e2732e443424195b9090081c5f70700"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "61aca29042d5a86a7469b124f6f942d9"
server_key_iv: "f2df4dc09175ae5f9d418338f8a4d38a"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.238701 22021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:40.268060 22021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:40.268819 22021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:40.269034 22021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:40.269718 22021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:40.270735 22021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:40.270813 22021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.270882 22021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:40.270932 22021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.280167 22021 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:38091
I20260808 10:43:40.280192 22155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:38091 every 8 connection(s)
I20260808 10:43:40.281090 22021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:40.286250 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22021
I20260808 10:43:40.286396 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:43:40.286752 1801 external_mini_cluster.cc:1428] Setting key 4b8688ba68ff82405e439b0edcd368f3
I20260808 10:43:40.289006 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42913
--builtin_ntp_servers=127.1.194.84:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:43:40.291728 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.283068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:43753 (local address 127.1.194.126:42913)
0808 10:43:40.283220 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.283223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.284301 (+ 1078us) server_negotiation.cc:407] Connection header received
0808 10:43:40.285690 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.285693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.285743 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.285825 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:40.287109 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.287759 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.289371 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.289687 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.289779 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.291148 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:40.291153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:40.291157 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:40.291174 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:40.291201 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:40.291212 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:40.291300 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.291459 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.291533 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:43:40.292707 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.283450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42913 (local address 127.1.194.66:43753)
0808 10:43:40.284148 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.284187 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.285447 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.286043 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.286055 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.286186 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:40.286902 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.286923 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.287896 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.287900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.289213 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.289228 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.289849 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.289856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.290090 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.290814 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:40.290817 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:40.290882 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:40.290980 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:40.291306 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:40.291313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:40.291318 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.291418 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.291680 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":128,"threads_started":1}
I20260808 10:43:40.294014 22156 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42913
I20260808 10:43:40.294257 22156 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:40.294811 22156 heartbeater.cc:511] Master 127.1.194.126:42913 requested a full tablet report, sending...
I20260808 10:43:40.295873 21822 ts_manager.cc:194] Registered new tserver with Master: 6e2732e443424195b9090081c5f70700 (127.1.194.66:38091)
I20260808 10:43:40.296475 21822 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.66:43753
W20260808 10:43:40.403676 22159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:40.403965 22159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:40.404066 22159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:40.408254 22159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:40.408334 22159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:40.408419 22159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:40.413177 22159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:40.414307 22159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:40.415774 22159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:40.425596 22165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:40.425606 22167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:40.425642 22159 server_base.cc:1034] running on GCE node
W20260808 10:43:40.425658 22164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:40.426363 22159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:40.427048 22159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:40.428265 22159 hybrid_clock.cc:648] HybridClock initialized: now 1786185820428230 us; error 46 us; skew 500 ppm
I20260808 10:43:40.430294 22159 webserver.cc:460] Webserver started at http://127.1.194.67:39947/ using document root <none> and password file <none>
I20260808 10:43:40.430908 22159 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:40.431007 22159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:40.431267 22159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:40.433528 22159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "eed96588858149bebac92f01385ade00"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "155b73ed9c79b3cfa93783a987513803"
server_key_iv: "75536aa1171901ee8b0b0e1330482cdb"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.434134 22159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "eed96588858149bebac92f01385ade00"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "155b73ed9c79b3cfa93783a987513803"
server_key_iv: "75536aa1171901ee8b0b0e1330482cdb"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.437990 22159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:43:40.440448 22173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.441807 22159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:40.441938 22159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "eed96588858149bebac92f01385ade00"
format_stamp: "Formatted at 2026-08-08 10:43:40 on dist-test-slave-ss4n"
server_key: "155b73ed9c79b3cfa93783a987513803"
server_key_iv: "75536aa1171901ee8b0b0e1330482cdb"
server_key_version: "encryptionkey@0"
I20260808 10:43:40.442058 22159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:40.473903 22159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:40.474707 22159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:40.474939 22159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:40.475590 22159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:40.476637 22159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:40.476720 22159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.476794 22159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:40.476830 22159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:40.486279 22159 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:35565
I20260808 10:43:40.486315 22289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:35565 every 8 connection(s)
I20260808 10:43:40.487260 22159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:40.487447 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22159
I20260808 10:43:40.487574 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:43:40.487938 1801 external_mini_cluster.cc:1428] Setting key 3f7159c7b65399e5831da983ad7b1229
I20260808 10:43:40.496821 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.489371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:54415 (local address 127.1.194.126:42913)
0808 10:43:40.489557 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.489562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.490918 (+ 1356us) server_negotiation.cc:407] Connection header received
0808 10:43:40.492269 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.492271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.492316 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.492384 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:40.493203 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.493816 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.494915 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.495124 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.495201 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.496317 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:40.496320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:40.496322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:40.496333 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:40.496351 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:40.496358 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:40.496419 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.496654 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.496705 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:43:40.497774 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.489917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42913 (local address 127.1.194.67:54415)
0808 10:43:40.490754 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.490808 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.492082 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.492507 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.492513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.492595 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:40.493071 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.493083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.493922 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.493924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.494801 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.494810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.495252 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.495255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.495394 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.496072 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:40.496074 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:40.496113 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:40.496173 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:40.496478 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:40.496487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:40.496494 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.496641 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.496908 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":140,"threads_started":1}
I20260808 10:43:40.498989 22290 heartbeater.cc:347] Connected to a master server at 127.1.194.126:42913
I20260808 10:43:40.499210 22290 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:40.499691 22290 heartbeater.cc:511] Master 127.1.194.126:42913 requested a full tablet report, sending...
I20260808 10:43:40.500831 21822 ts_manager.cc:194] Registered new tserver with Master: eed96588858149bebac92f01385ade00 (127.1.194.67:35565)
I20260808 10:43:40.501592 21822 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.67:54415
I20260808 10:43:40.502660 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:43:40.508464 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.504696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.1.194.126:42913)
0808 10:43:40.504822 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.504826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.505015 (+ 189us) server_negotiation.cc:407] Connection header received
0808 10:43:40.505114 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.505116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.505152 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.505220 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:40.505469 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.506006 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.506816 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.507018 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.507111 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.508076 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:40.508079 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:40.508081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:40.508093 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:40.508111 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:40.508118 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:40.508230 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.508330 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.508363 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:43:40.511408 21822 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:43:40.515022 21822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:43:40.530139 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.524443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:35565 (local address 127.0.0.1:52884)
0808 10:43:40.524681 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.524695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.524779 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.526294 (+ 1515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.526297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.526315 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:40.526397 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.526405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.527668 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.527671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.528782 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.528791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.529623 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.529626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.529777 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.529800 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.529866 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.529941 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":26,"thread_start_us":92,"threads_started":1}
I20260808 10:43:40.530187 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.525173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.1.194.67:35565)
0808 10:43:40.525893 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.525906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.525929 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:43:40.526019 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.526023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.526151 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.526316 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:40.526561 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.527517 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.528900 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.529455 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.529701 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.529885 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.529935 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.530029 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":204,"threads_started":1}
I20260808 10:43:40.530524 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.523703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40549 (local address 127.0.0.1:49352)
0808 10:43:40.524242 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.524290 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.524431 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.525399 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.525405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.525430 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:40.525611 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.525618 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.527050 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.527054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.528371 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.528380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.529435 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.529439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.529601 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.529689 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.529793 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.529913 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":97,"thread_start_us":84,"threads_started":1}
I20260808 10:43:40.530673 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.524989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.1.194.66:38091)
0808 10:43:40.525451 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.525467 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.525514 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:43:40.525591 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.525596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.525787 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.525925 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:40.526302 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.527302 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.528824 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.529597 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.529703 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.529863 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.530386 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.530509 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":151,"threads_started":1}
I20260808 10:43:40.530841 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.523869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49352 (local address 127.1.194.65:40549)
0808 10:43:40.524736 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.524750 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.524776 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:43:40.524856 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.524860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.525060 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.525217 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:40.525761 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.526923 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.528507 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.529297 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.529416 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.529640 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.530018 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.530108 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":120,"threads_started":1}
I20260808 10:43:40.530925 22301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.523890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38091 (local address 127.0.0.1:53670)
0808 10:43:40.524319 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.524336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.524433 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.526060 (+ 1627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.526063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.526080 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:40.526170 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.526176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.527418 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.527421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.528680 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.528690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.529922 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.529926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.530007 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.530027 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.530093 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.530150 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":36,"thread_start_us":94,"threads_started":1}
I20260808 10:43:40.533540 22087 tablet_service.cc:1467] Processing CreateTablet for tablet 111895689bb843f294ab50584551f97e (DEFAULT_TABLE table=test-table [id=e9428ef9a27245859d962ff9d3b1addc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:40.533547 22223 tablet_service.cc:1467] Processing CreateTablet for tablet 111895689bb843f294ab50584551f97e (DEFAULT_TABLE table=test-table [id=e9428ef9a27245859d962ff9d3b1addc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:40.533546 21949 tablet_service.cc:1467] Processing CreateTablet for tablet 111895689bb843f294ab50584551f97e (DEFAULT_TABLE table=test-table [id=e9428ef9a27245859d962ff9d3b1addc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:40.534611 22087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 111895689bb843f294ab50584551f97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:40.534616 22223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 111895689bb843f294ab50584551f97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:40.534667 21949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 111895689bb843f294ab50584551f97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:40.541353 22307 tablet_bootstrap.cc:492] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Bootstrap starting.
I20260808 10:43:40.542147 22309 tablet_bootstrap.cc:492] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: Bootstrap starting.
I20260808 10:43:40.543704 22307 tablet_bootstrap.cc:654] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:40.543926 22309 tablet_bootstrap.cc:654] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:40.544473 22307 log.cc:824] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:40.544675 22309 log.cc:824] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:40.545183 22308 tablet_bootstrap.cc:492] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Bootstrap starting.
I20260808 10:43:40.546792 22307 tablet_bootstrap.cc:492] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: No bootstrap required, opened a new log
I20260808 10:43:40.546979 22307 ts_tablet_manager.cc:1397] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:43:40.547147 22309 tablet_bootstrap.cc:492] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: No bootstrap required, opened a new log
I20260808 10:43:40.547343 22309 ts_tablet_manager.cc:1397] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:40.547437 22308 tablet_bootstrap.cc:654] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:40.548246 22308 log.cc:824] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:40.550239 22308 tablet_bootstrap.cc:492] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: No bootstrap required, opened a new log
I20260808 10:43:40.550419 22308 ts_tablet_manager.cc:1397] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:40.551146 22309 raft_consensus.cc:348] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.551386 22309 raft_consensus.cc:374] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:40.551440 22309 raft_consensus.cc:729] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e2e65eff4b456c906a7337e8feeb44, State: Initialized, Role: FOLLOWER
I20260808 10:43:40.551350 22307 raft_consensus.cc:348] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.551604 22307 raft_consensus.cc:374] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:40.551664 22307 raft_consensus.cc:729] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2732e443424195b9090081c5f70700, State: Initialized, Role: FOLLOWER
I20260808 10:43:40.551956 22309 consensus_queue.cc:260] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.552076 22307 consensus_queue.cc:260] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.552748 22156 heartbeater.cc:503] Master 127.1.194.126:42913 was elected leader, sending a full tablet report...
I20260808 10:43:40.552943 22309 ts_tablet_manager.cc:1428] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:40.553006 22307 ts_tablet_manager.cc:1428] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:40.553448 22018 heartbeater.cc:503] Master 127.1.194.126:42913 was elected leader, sending a full tablet report...
I20260808 10:43:40.554504 22308 raft_consensus.cc:348] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.554762 22308 raft_consensus.cc:374] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:40.554821 22308 raft_consensus.cc:729] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed96588858149bebac92f01385ade00, State: Initialized, Role: FOLLOWER
I20260808 10:43:40.555233 22308 consensus_queue.cc:260] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.556165 22308 ts_tablet_manager.cc:1428] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:40.556231 22290 heartbeater.cc:503] Master 127.1.194.126:42913 was elected leader, sending a full tablet report...
W20260808 10:43:40.582515 22019 tablet.cc:2369] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:43:40.638592 22315 raft_consensus.cc:482] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:40.638835 22315 raft_consensus.cc:504] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.640110 22315 leader_election.cc:288] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0e2e65eff4b456c906a7337e8feeb44 (127.1.194.65:40549), 6e2732e443424195b9090081c5f70700 (127.1.194.66:38091)
I20260808 10:43:40.644254 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.640323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40549 (local address 127.1.194.67:39011)
0808 10:43:40.640442 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.640460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.640572 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.640927 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.640931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.640958 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:40.641080 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.641088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.642015 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.642019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.643039 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.643049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.643794 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.643797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.643911 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.643936 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.644010 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.644086 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":43}
I20260808 10:43:40.644273 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.640531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:39011 (local address 127.1.194.65:40549)
0808 10:43:40.640668 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.640672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.640687 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:43:40.640742 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.640744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.640793 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.640881 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:40.641232 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.641905 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.643160 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.643671 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.643766 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.643802 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.644063 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.644140 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:43:40.644416 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.640503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:57633 (local address 127.1.194.66:38091)
0808 10:43:40.640653 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.640659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.640931 (+ 272us) server_negotiation.cc:407] Connection header received
0808 10:43:40.641032 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.641037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.641090 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.641181 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:40.641520 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.642124 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.643392 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.643827 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.643942 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.644001 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.644213 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.644293 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:43:40.644486 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.640398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:38091 (local address 127.1.194.67:57633)
0808 10:43:40.640838 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.640853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:40.640941 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:40.641218 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:40.641221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:40.641237 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:40.641369 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.641375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.642252 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.642254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.643272 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:40.643282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.643950 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.643953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:40.644079 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.644101 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:43:40.644179 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:43:40.644241 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":37,"thread_start_us":137,"threads_started":1}
I20260808 10:43:40.645061 21969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "111895689bb843f294ab50584551f97e" candidate_uuid: "eed96588858149bebac92f01385ade00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e2e65eff4b456c906a7337e8feeb44" is_pre_election: true
I20260808 10:43:40.645190 22107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "111895689bb843f294ab50584551f97e" candidate_uuid: "eed96588858149bebac92f01385ade00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2732e443424195b9090081c5f70700" is_pre_election: true
I20260808 10:43:40.645395 21969 raft_consensus.cc:2454] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eed96588858149bebac92f01385ade00 in term 0.
I20260808 10:43:40.645507 22107 raft_consensus.cc:2454] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eed96588858149bebac92f01385ade00 in term 0.
I20260808 10:43:40.645987 22177 leader_election.cc:302] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6e2732e443424195b9090081c5f70700, a0e2e65eff4b456c906a7337e8feeb44, eed96588858149bebac92f01385ade00; no voters:
I20260808 10:43:40.646281 22315 raft_consensus.cc:2781] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:43:40.646374 22315 raft_consensus.cc:482] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:43:40.646418 22315 raft_consensus.cc:3037] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:40.647456 22315 raft_consensus.cc:504] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.647814 22315 leader_election.cc:288] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [CANDIDATE]: Term 1 election: Requested vote from peers a0e2e65eff4b456c906a7337e8feeb44 (127.1.194.65:40549), 6e2732e443424195b9090081c5f70700 (127.1.194.66:38091)
I20260808 10:43:40.648132 21969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "111895689bb843f294ab50584551f97e" candidate_uuid: "eed96588858149bebac92f01385ade00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e2e65eff4b456c906a7337e8feeb44"
I20260808 10:43:40.648176 22107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "111895689bb843f294ab50584551f97e" candidate_uuid: "eed96588858149bebac92f01385ade00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2732e443424195b9090081c5f70700"
I20260808 10:43:40.648242 21969 raft_consensus.cc:3037] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:40.648316 22107 raft_consensus.cc:3037] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:40.649294 21969 raft_consensus.cc:2454] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eed96588858149bebac92f01385ade00 in term 1.
I20260808 10:43:40.649467 22107 raft_consensus.cc:2454] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eed96588858149bebac92f01385ade00 in term 1.
I20260808 10:43:40.649665 22175 leader_election.cc:302] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0e2e65eff4b456c906a7337e8feeb44, eed96588858149bebac92f01385ade00; no voters:
I20260808 10:43:40.649857 22315 raft_consensus.cc:2781] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:40.650127 22315 raft_consensus.cc:686] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 1 LEADER]: Becoming Leader. State: Replica: eed96588858149bebac92f01385ade00, State: Running, Role: LEADER
I20260808 10:43:40.650456 22315 consensus_queue.cc:237] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } }
I20260808 10:43:40.653709 21822 catalog_manager.cc:5495] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 reported cstate change: term changed from 0 to 1, leader changed from <none> to eed96588858149bebac92f01385ade00 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "eed96588858149bebac92f01385ade00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eed96588858149bebac92f01385ade00" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 35565 } health_report { overall_health: HEALTHY } } }
I20260808 10:43:40.684912 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.681157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.1.194.67:35565)
0808 10:43:40.681307 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:40.681312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:40.681327 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:43:40.681456 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:40.681458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:40.681533 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:40.681709 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:43:40.681969 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.682479 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.683384 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:40.683614 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:40.683738 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:40.684054 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:43:40.684455 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:43:40.684627 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:40.684713 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 10:43:40.684748 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:43:40.690768 21969 raft_consensus.cc:1261] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 1 FOLLOWER]: Refusing update from remote peer eed96588858149bebac92f01385ade00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:40.690768 22107 raft_consensus.cc:1261] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 1 FOLLOWER]: Refusing update from remote peer eed96588858149bebac92f01385ade00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:40.691531 22317 consensus_queue.cc:1035] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e2e65eff4b456c906a7337e8feeb44" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 40549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:40.691748 22315 consensus_queue.cc:1035] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2732e443424195b9090081c5f70700" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:40.701298 22322 mvcc.cc:205] Tried to move back new op lower bound from 7316217121541984256 to 7316217121386786816. Current Snapshot: MvccSnapshot[applied={T|T < 7316217121541984256}]
I20260808 10:43:40.708722 21822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:40.708918 21822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:40.711606 21822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c00ee9750b543ff83a2fd5b4f83a808: Sending DeleteTablet for 3 replicas of tablet 111895689bb843f294ab50584551f97e
I20260808 10:43:40.712473 21949 tablet_service.cc:1514] Processing DeleteTablet for tablet 111895689bb843f294ab50584551f97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:40 UTC) from {username='slave'} at 127.0.0.1:49352
I20260808 10:43:40.712498 22087 tablet_service.cc:1514] Processing DeleteTablet for tablet 111895689bb843f294ab50584551f97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:40 UTC) from {username='slave'} at 127.0.0.1:53670
I20260808 10:43:40.713078 22330 tablet_replica.cc:330] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: stopping tablet replica
I20260808 10:43:40.713078 22331 tablet_replica.cc:330] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44: stopping tablet replica
I20260808 10:43:40.713074 22223 tablet_service.cc:1514] Processing DeleteTablet for tablet 111895689bb843f294ab50584551f97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:40 UTC) from {username='slave'} at 127.0.0.1:52884
I20260808 10:43:40.713421 22330 raft_consensus.cc:2229] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:40.713449 22331 raft_consensus.cc:2229] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:40.713812 22331 raft_consensus.cc:2258] T 111895689bb843f294ab50584551f97e P a0e2e65eff4b456c906a7337e8feeb44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:40.713812 22330 raft_consensus.cc:2258] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:40.714347 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21885
I20260808 10:43:40.716552 22330 ts_tablet_manager.cc:1905] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:40.719437 22330 ts_tablet_manager.cc:1918] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:40.719539 22332 tablet_replica.cc:330] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: stopping tablet replica
I20260808 10:43:40.719676 22330 log.cc:1196] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/111895689bb843f294ab50584551f97e
I20260808 10:43:40.719888 22332 raft_consensus.cc:2229] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:43:40.720151 22330 ts_tablet_manager.cc:1939] T 111895689bb843f294ab50584551f97e P 6e2732e443424195b9090081c5f70700: Deleting consensus metadata
I20260808 10:43:40.720377 22332 raft_consensus.cc:2258] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:40.721596 21809 catalog_manager.cc:4841] TS 6e2732e443424195b9090081c5f70700 (127.1.194.66:38091): tablet 111895689bb843f294ab50584551f97e (table test-table [id=e9428ef9a27245859d962ff9d3b1addc]) successfully deleted
I20260808 10:43:40.721937 22332 ts_tablet_manager.cc:1905] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:43:40.724076 21807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:40549 (error 108)
I20260808 10:43:40.724393 22332 ts_tablet_manager.cc:1918] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:40.724496 22332 log.cc:1196] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/111895689bb843f294ab50584551f97e
I20260808 10:43:40.724835 22332 ts_tablet_manager.cc:1939] T 111895689bb843f294ab50584551f97e P eed96588858149bebac92f01385ade00: Deleting consensus metadata
I20260808 10:43:40.724893 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22021
I20260808 10:43:40.726241 21809 catalog_manager.cc:4841] TS eed96588858149bebac92f01385ade00 (127.1.194.67:35565): tablet 111895689bb843f294ab50584551f97e (table test-table [id=e9428ef9a27245859d962ff9d3b1addc]) successfully deleted
I20260808 10:43:40.727919 22301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:40.727588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:40549 (local address 127.0.0.1:49364)
0808 10:43:40.727732 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:40.727814 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:40549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 10:43:40.728179 21807 catalog_manager.cc:4584] TS a0e2e65eff4b456c906a7337e8feeb44 (127.1.194.65:40549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 111895689bb843f294ab50584551f97e: Network error: Client connection negotiation failed: client connection to 127.1.194.65:40549: connect: Connection refused (error 111)
I20260808 10:43:40.732854 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22159
I20260808 10:43:40.739591 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 21791
2026-08-08T10:43:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1104 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:40 dist-test-slave-ss4n krb5kdc[22336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:40 dist-test-slave-ss4n krb5kdc[22336](info): set up 2 sockets
Aug 08 10:43:40 dist-test-slave-ss4n krb5kdc[22336](info): commencing operation
krb5kdc: starting...
W20260808 10:43:41.834504 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:41 dist-test-slave-ss4n krb5kdc[22336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:43:41.978404 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45907
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:42.091133 22350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:42.091465 22350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:42.091545 22350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:42.095623 22350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:42.095710 22350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:42.095747 22350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:42.095803 22350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:42.095849 22350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:42.101151 22350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:42.102262 22350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:42.103286 22350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:42.112233 22358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.112244 22356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.112275 22355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:42.112496 22350 server_base.cc:1034] running on GCE node
I20260808 10:43:42.113121 22350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:42.114285 22350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:42.115442 22350 hybrid_clock.cc:648] HybridClock initialized: now 1786185822115424 us; error 60 us; skew 500 ppm
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:42.118391 22350 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:43:42.119275 22350 webserver.cc:460] Webserver started at http://127.1.194.126:44489/ using document root <none> and password file <none>
I20260808 10:43:42.119822 22350 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:42.119884 22350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:42.120067 22350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:42.122112 22350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "4162b239b577423ab1abdd952f0b6674"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "41e43dd53e56b507c835b90708de6654"
server_key_iv: "042b51c3a4f721416b289256a361deff"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.122632 22350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "4162b239b577423ab1abdd952f0b6674"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "41e43dd53e56b507c835b90708de6654"
server_key_iv: "042b51c3a4f721416b289256a361deff"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.126247 22350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:43:42.128531 22365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.129804 22350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:42.129943 22350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "4162b239b577423ab1abdd952f0b6674"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "41e43dd53e56b507c835b90708de6654"
server_key_iv: "042b51c3a4f721416b289256a361deff"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.130069 22350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:42.140816 22350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:42.144057 22350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:42.144268 22350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:42.151925 22350 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:45907
I20260808 10:43:42.151964 22418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:45907 every 8 connection(s)
I20260808 10:43:42.152904 22350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:42.155486 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22350
I20260808 10:43:42.155663 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:42.155961 1801 external_mini_cluster.cc:1428] Setting key 6bce17ff147c9f2de21f932d22f44c7e
I20260808 10:43:42.156172 22419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:42.164602 22419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674: Bootstrap starting.
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:42.167129 22419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:42.167913 22419 log.cc:824] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:42.170385 22419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674: No bootstrap required, opened a new log
I20260808 10:43:42.173228 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.157590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.1.194.126:45907)
0808 10:43:42.158049 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.158069 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.158113 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:43:42.159258 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.159285 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.159736 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.160152 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:42.160912 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.162625 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.163667 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.164408 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.164585 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.167331 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:42.167336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:42.167350 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:42.167379 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:42.169693 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.170186 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.170192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.170198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.170342 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.170740 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.170743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.170745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.171292 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:42.171488 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.171796 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.172123 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260808 10:43:42.175254 22419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4162b239b577423ab1abdd952f0b6674" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45907 } }
I20260808 10:43:42.175587 22419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:42.175691 22419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4162b239b577423ab1abdd952f0b6674, State: Initialized, Role: FOLLOWER
I20260808 10:43:42.176080 22419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4162b239b577423ab1abdd952f0b6674" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45907 } }
I20260808 10:43:42.176198 22419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:42.176245 22419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:42.176344 22419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:42.177286 22419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4162b239b577423ab1abdd952f0b6674" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45907 } }
I20260808 10:43:42.177680 22419 leader_election.cc:302] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4162b239b577423ab1abdd952f0b6674; no voters:
I20260808 10:43:42.178007 22419 leader_election.cc:288] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:42.178148 22424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:42.178392 22424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [term 1 LEADER]: Becoming Leader. State: Replica: 4162b239b577423ab1abdd952f0b6674, State: Running, Role: LEADER
I20260808 10:43:42.178728 22424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4162b239b577423ab1abdd952f0b6674" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45907 } }
I20260808 10:43:42.179528 22419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:42.180528 22426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4162b239b577423ab1abdd952f0b6674. Latest consensus state: current_term: 1 leader_uuid: "4162b239b577423ab1abdd952f0b6674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4162b239b577423ab1abdd952f0b6674" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45907 } } }
I20260808 10:43:42.180691 22426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:42.180527 22425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4162b239b577423ab1abdd952f0b6674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4162b239b577423ab1abdd952f0b6674" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45907 } } }
I20260808 10:43:42.180965 22425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:42.181186 22433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:42.184424 22433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:42.189582 22433 catalog_manager.cc:1347] Generated new cluster ID: 79223ee6c4814cc697c7c1d13861d8de
I20260808 10:43:42.189652 22433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:42.197198 22433 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:42.198575 22433 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:42.209214 22433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674: Generated new TSK 0
I20260808 10:43:42.210093 22433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.194.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
Entry for principal kudu/127.1.194.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab.
I20260808 10:43:42.242141 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45907
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:42.359581 22445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:42.359858 22445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:42.359903 22445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:42.364455 22445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:42.364547 22445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:42.364640 22445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:42.370246 22445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.65.keytab
--principal=kudu/127.1.194.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:42.371297 22445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:42.372303 22445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:42.382182 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.382182 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.382236 22453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:42.382579 22445 server_base.cc:1034] running on GCE node
I20260808 10:43:42.383001 22445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:42.383648 22445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:42.384869 22445 hybrid_clock.cc:648] HybridClock initialized: now 1786185822384846 us; error 37 us; skew 500 ppm
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:42.387786 22445 init.cc:377] Logged in from keytab as kudu/127.1.194.65@KRBTEST.COM (short username kudu)
I20260808 10:43:42.388687 22445 webserver.cc:460] Webserver started at http://127.1.194.65:38897/ using document root <none> and password file <none>
I20260808 10:43:42.389287 22445 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:42.389389 22445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:42.389659 22445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:42.391736 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "d80dc5c238a04b22a80edd43c39aa188"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "bc9742130e346a6197803a88a8cedc8b"
server_key_iv: "417907a5b12b56a965599370228f0281"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.392350 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "d80dc5c238a04b22a80edd43c39aa188"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "bc9742130e346a6197803a88a8cedc8b"
server_key_iv: "417907a5b12b56a965599370228f0281"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.396427 22445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:43:42.399139 22461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.400368 22445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:43:42.400522 22445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "d80dc5c238a04b22a80edd43c39aa188"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "bc9742130e346a6197803a88a8cedc8b"
server_key_iv: "417907a5b12b56a965599370228f0281"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.400648 22445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:42.415613 22445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:42.419484 22445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:42.419798 22445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:42.420431 22445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:42.421298 22445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:42.421376 22445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.421444 22445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:42.421501 22445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.430337 22445 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:42591
I20260808 10:43:42.430413 22576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:42591 every 8 connection(s)
I20260808 10:43:42.431267 22445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:43:42.439810 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22445
I20260808 10:43:42.440011 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:43:42.440333 1801 external_mini_cluster.cc:1428] Setting key 96bd6839241e404bbdaa10a282e4f6a1
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.65@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:42.446174 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.433222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:51895 (local address 127.1.194.126:45907)
0808 10:43:42.433361 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.433365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.434502 (+ 1137us) server_negotiation.cc:407] Connection header received
0808 10:43:42.435466 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.435470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.435524 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.435637 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:42.436888 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.437445 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.438663 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.438902 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.439032 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.441922 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:42.441926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:42.441928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:42.441957 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:42.443570 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.444197 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.444202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.444204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.444308 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.444769 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.444773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.444775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.445043 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:42.445155 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.445767 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.445916 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:43:42.446952 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.433701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45907 (local address 127.1.194.65:51895)
0808 10:43:42.434304 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:42.434387 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:42.435258 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:42.435815 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:42.435821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:42.436172 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:42.436734 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.436749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.437635 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.437638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.438511 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.438521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.439063 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.439067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.439251 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.439906 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:42.439922 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:42.441744 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:42.443704 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:42.443711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:42.443729 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:42.444087 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:42.444443 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:42.444446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:42.444448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:42.444658 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:42.445159 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:42.445164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:42.445398 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:43:42.445690 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 10:43:42.445965 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":119,"threads_started":1}
I20260808 10:43:42.448323 22577 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45907
I20260808 10:43:42.448606 22577 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:42.449239 22577 heartbeater.cc:511] Master 127.1.194.126:45907 requested a full tablet report, sending...
I20260808 10:43:42.450940 22383 ts_manager.cc:194] Registered new tserver with Master: d80dc5c238a04b22a80edd43c39aa188 (127.1.194.65:42591)
WARNING: no policy specified for kudu/127.1.194.66@KRBTEST.COM; defaulting to no policy
I20260808 10:43:42.452111 22383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.65@KRBTEST.COM'} at 127.1.194.65:51895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
Entry for principal kudu/127.1.194.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab.
I20260808 10:43:42.473829 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45907
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:42.590025 22582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:42.590305 22582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:42.590384 22582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:42.594437 22582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:42.594530 22582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:42.594621 22582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:42.599792 22582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.66.keytab
--principal=kudu/127.1.194.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:42.600917 22582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:42.601974 22582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:42.611820 22588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.611842 22590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.611867 22587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:42.611908 22582 server_base.cc:1034] running on GCE node
I20260808 10:43:42.612567 22582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:42.613263 22582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:42.614431 22582 hybrid_clock.cc:648] HybridClock initialized: now 1786185822614417 us; error 69 us; skew 500 ppm
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:42.617753 22582 init.cc:377] Logged in from keytab as kudu/127.1.194.66@KRBTEST.COM (short username kudu)
I20260808 10:43:42.618675 22582 webserver.cc:460] Webserver started at http://127.1.194.66:43435/ using document root <none> and password file <none>
I20260808 10:43:42.619283 22582 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:42.619390 22582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:42.619648 22582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:42.621933 22582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "d93fb7d29577467ab6781cd83c6f8db7"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "b383d4503d3794afbfdd2a18a1ea89eb"
server_key_iv: "f984a9686e318fb45bbb6f3573754907"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.622563 22582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "d93fb7d29577467ab6781cd83c6f8db7"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "b383d4503d3794afbfdd2a18a1ea89eb"
server_key_iv: "f984a9686e318fb45bbb6f3573754907"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.626389 22582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:43:42.628950 22597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.630368 22582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:42.630518 22582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "d93fb7d29577467ab6781cd83c6f8db7"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "b383d4503d3794afbfdd2a18a1ea89eb"
server_key_iv: "f984a9686e318fb45bbb6f3573754907"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.630650 22582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:42.648175 22582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:42.652016 22582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:42.652446 22582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:42.653151 22582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:42.654191 22582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:42.654275 22582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.654351 22582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:42.654407 22582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.664206 22582 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:33637
I20260808 10:43:42.664225 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:33637 every 8 connection(s)
I20260808 10:43:42.665210 22582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:42.671417 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22582
I20260808 10:43:42.671597 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:43:42.671958 1801 external_mini_cluster.cc:1428] Setting key 99a9fe7a171dbe8595f700328bc0a3c1
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.66@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:42.680542 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.667224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:49873 (local address 127.1.194.126:45907)
0808 10:43:42.667407 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.667412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.668280 (+ 868us) server_negotiation.cc:407] Connection header received
0808 10:43:42.669201 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.669204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.669253 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.669332 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:42.670514 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.671041 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.672693 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.672939 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.673052 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.676349 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:42.676355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:42.676361 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:42.676393 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:42.678092 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.678689 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.678694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.678696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.678788 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.679197 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.679202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.679204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.679414 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:42.679548 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.680205 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.680343 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 10:43:42.681584 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.667667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45907 (local address 127.1.194.66:49873)
0808 10:43:42.668120 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:42.668162 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:42.668928 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:42.669543 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:42.669554 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:42.669920 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:42.670378 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.670391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.671195 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.671201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.672506 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.672543 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.673103 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.673107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.673323 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.673987 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:42.674008 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:42.676096 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:42.678235 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:42.678242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:42.678259 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:42.678563 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:42.678896 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:42.678901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:42.678907 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:42.679083 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:42.679536 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:42.679543 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:42.679913 (+ 370us) client_negotiation.cc:769] Sending connection context
0808 10:43:42.680133 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 10:43:42.680492 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.1.194.67@KRBTEST.COM; defaulting to no policy
I20260808 10:43:42.682988 22714 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45907
I20260808 10:43:42.683292 22714 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:42.683820 22714 heartbeater.cc:511] Master 127.1.194.126:45907 requested a full tablet report, sending...
I20260808 10:43:42.684904 22383 ts_manager.cc:194] Registered new tserver with Master: d93fb7d29577467ab6781cd83c6f8db7 (127.1.194.66:33637)
I20260808 10:43:42.685669 22383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.66@KRBTEST.COM'} at 127.1.194.66:49873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
Entry for principal kudu/127.1.194.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab.
I20260808 10:43:42.704802 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45907
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:41295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:42.818192 22719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:42.818454 22719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:42.818496 22719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:42.822551 22719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:42.822614 22719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:42.822674 22719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:42.827842 22719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:41295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.67.keytab
--principal=kudu/127.1.194.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:42.828814 22719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:42.829770 22719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:42.839815 22725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:42.839903 22719 server_base.cc:1034] running on GCE node
W20260808 10:43:42.839828 22724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:42.839831 22727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:42.840461 22719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:42.841048 22719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:42.842243 22719 hybrid_clock.cc:648] HybridClock initialized: now 1786185822842230 us; error 27 us; skew 500 ppm
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:42.845036 22719 init.cc:377] Logged in from keytab as kudu/127.1.194.67@KRBTEST.COM (short username kudu)
I20260808 10:43:42.846050 22719 webserver.cc:460] Webserver started at http://127.1.194.67:43199/ using document root <none> and password file <none>
I20260808 10:43:42.846643 22719 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:42.846742 22719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:42.846964 22719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:42.849020 22719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "0a35523d6fe4436eb10289f3045fdf9c"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "c8d12b2b3408ffd6390b50d0641109dc"
server_key_iv: "73014eada7083cf4ccef9d24f84d0457"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.849680 22719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "0a35523d6fe4436eb10289f3045fdf9c"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "c8d12b2b3408ffd6390b50d0641109dc"
server_key_iv: "73014eada7083cf4ccef9d24f84d0457"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.853384 22719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:42.855774 22734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.856974 22719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:42.857117 22719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "0a35523d6fe4436eb10289f3045fdf9c"
format_stamp: "Formatted at 2026-08-08 10:43:42 on dist-test-slave-ss4n"
server_key: "c8d12b2b3408ffd6390b50d0641109dc"
server_key_iv: "73014eada7083cf4ccef9d24f84d0457"
server_key_version: "encryptionkey@0"
I20260808 10:43:42.857230 22719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:42.869982 22719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:42.872781 22719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:42.872996 22719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:42.873646 22719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:42.874449 22719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:42.874524 22719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.874603 22719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:42.874660 22719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:42.883876 22719 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:39135
I20260808 10:43:42.883901 22849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:39135 every 8 connection(s)
I20260808 10:43:42.884752 22719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:42.892940 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22719
I20260808 10:43:42.893121 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:43:42.893528 1801 external_mini_cluster.cc:1428] Setting key e2fb01011e22d5fc13217afa4e3b23f6
Aug 08 10:43:42 dist-test-slave-ss4n krb5kdc[22336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185822, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.67@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:43:42.898453 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.886676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:51473 (local address 127.1.194.126:45907)
0808 10:43:42.886835 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.886839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.887689 (+ 850us) server_negotiation.cc:407] Connection header received
0808 10:43:42.888500 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.888503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.888551 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.888633 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:42.889780 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.890301 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.891427 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.891657 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.891738 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.894564 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:42.894568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:42.894576 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:42.894632 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:42.896375 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.896911 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.896915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.896917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.897002 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.897314 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.897317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.897318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.897546 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:42.897630 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.898113 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.898263 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:43:42.899447 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.887029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:45907 (local address 127.1.194.67:51473)
0808 10:43:42.887511 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:42.887565 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:42.888309 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:42.888790 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:42.888796 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:42.889167 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:42.889650 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.889664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.890412 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.890415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.891312 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.891321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.891790 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.891793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.891948 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.892615 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:43:42.892631 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:42.894387 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:42.896486 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:42.896491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:42.896503 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:42.896804 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:42.897089 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:43:42.897091 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:43:42.897092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:43:42.897231 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:43:42.897651 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:42.897656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:42.897892 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 10:43:42.898060 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 10:43:42.898471 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":112,"threads_started":1}
I20260808 10:43:42.900569 22850 heartbeater.cc:347] Connected to a master server at 127.1.194.126:45907
I20260808 10:43:42.900807 22850 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:42.901250 22850 heartbeater.cc:511] Master 127.1.194.126:45907 requested a full tablet report, sending...
I20260808 10:43:42.902308 22383 ts_manager.cc:194] Registered new tserver with Master: 0a35523d6fe4436eb10289f3045fdf9c (127.1.194.67:39135)
I20260808 10:43:42.902839 22383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.194.67@KRBTEST.COM'} at 127.1.194.67:51473
I20260808 10:43:42.907915 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:43:42.918265 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.910090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.1.194.126:45907)
0808 10:43:42.910241 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.910246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.910451 (+ 205us) server_negotiation.cc:407] Connection header received
0808 10:43:42.910645 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.910647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.910689 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.910781 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:42.911518 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.912039 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.912915 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.913141 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.913252 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.914819 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:42.914823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:42.914826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:42.914852 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:42.916458 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.916889 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.916893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.916895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.916978 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:42.917271 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:42.917275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:42.917277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:42.917571 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:42.917726 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.917977 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.918103 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:43:42.921383 22383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:43:42.924485 22383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:43:42.939518 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.932911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.1.194.65:42591)
0808 10:43:42.933568 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.934010 (+ 442us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.934041 (+ 31us) server_negotiation.cc:407] Connection header received
0808 10:43:42.934126 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.934131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.934472 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.934960 (+ 488us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:42.935130 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.936080 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.937945 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.938785 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.938914 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.939064 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.939291 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.939393 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":94,"threads_started":1}
I20260808 10:43:42.939488 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.932686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:42591 (local address 127.0.0.1:51688)
0808 10:43:42.933156 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:42.933209 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:42.933341 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:42.934638 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:42.934642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:42.934671 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:42.934807 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.934814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.936212 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.936218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.937818 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.937828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.938923 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.938930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.939039 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.939075 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:43:42.939210 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:43:42.939336 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":134,"spinlock_wait_cycles":768,"thread_start_us":112,"threads_started":1}
I20260808 10:43:42.939728 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.933801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.1.194.66:33637)
0808 10:43:42.934543 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.934555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.934804 (+ 249us) server_negotiation.cc:407] Connection header received
0808 10:43:42.934888 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.934896 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.935068 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.935222 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:42.935658 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.936721 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.938325 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.938904 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.939007 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.939140 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.939480 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.939587 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":141,"threads_started":1}
I20260808 10:43:42.939779 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.933692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.1.194.67:39135)
0808 10:43:42.934075 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:42.934097 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:42.934127 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:43:42.934191 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:42.934195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:42.934397 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:42.934564 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:42.934932 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.936145 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.938101 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.938839 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.938930 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.939064 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:42.939566 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 10:43:42.939648 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":156,"threads_started":1}
I20260808 10:43:42.939991 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.933229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:39135 (local address 127.0.0.1:48702)
0808 10:43:42.933521 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:42.933538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:42.933630 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:42.934574 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:42.934583 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:42.934612 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:42.934776 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.934784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.936307 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.936310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.937970 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.937981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.939134 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.939138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.939269 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.939346 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 10:43:42.939416 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:43:42.939493 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":223,"spinlock_wait_cycles":2304,"thread_start_us":122,"threads_started":1}
I20260808 10:43:42.940106 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:42.934156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:33637 (local address 127.0.0.1:45320)
0808 10:43:42.934687 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:42.934701 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:42.934794 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:42.935402 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:42.935406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:42.935424 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:42.935531 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.935541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.936963 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.936967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.938208 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:42.938215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:42.939134 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:42.939138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:42.939305 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:42.939342 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:43:42.939882 (+ 540us) client_negotiation.cc:240] Negotiation successful
0808 10:43:42.939954 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":108,"thread_start_us":97,"threads_started":1}
I20260808 10:43:42.942816 22784 tablet_service.cc:1467] Processing CreateTablet for tablet f8ac123a0b4c4a0cb29bfaca13add06c (DEFAULT_TABLE table=test-table [id=ee0222068ec64871bcfd80644ed61b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:42.942816 22510 tablet_service.cc:1467] Processing CreateTablet for tablet f8ac123a0b4c4a0cb29bfaca13add06c (DEFAULT_TABLE table=test-table [id=ee0222068ec64871bcfd80644ed61b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:42.943768 22510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ac123a0b4c4a0cb29bfaca13add06c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:42.943774 22784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ac123a0b4c4a0cb29bfaca13add06c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:42.949853 22647 tablet_service.cc:1467] Processing CreateTablet for tablet f8ac123a0b4c4a0cb29bfaca13add06c (DEFAULT_TABLE table=test-table [id=ee0222068ec64871bcfd80644ed61b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:42.950413 22868 tablet_bootstrap.cc:492] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Bootstrap starting.
I20260808 10:43:42.951011 22647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ac123a0b4c4a0cb29bfaca13add06c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:42.952299 22868 tablet_bootstrap.cc:654] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:42.953259 22868 log.cc:824] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:42.953857 22867 tablet_bootstrap.cc:492] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: Bootstrap starting.
I20260808 10:43:42.955348 22868 tablet_bootstrap.cc:492] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: No bootstrap required, opened a new log
I20260808 10:43:42.955631 22868 ts_tablet_manager.cc:1397] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260808 10:43:42.955864 22867 tablet_bootstrap.cc:654] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:42.956562 22867 log.cc:824] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:42.957279 22871 tablet_bootstrap.cc:492] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Bootstrap starting.
I20260808 10:43:42.958586 22867 tablet_bootstrap.cc:492] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: No bootstrap required, opened a new log
I20260808 10:43:42.958772 22867 ts_tablet_manager.cc:1397] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:42.959354 22871 tablet_bootstrap.cc:654] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:42.959285 22868 raft_consensus.cc:348] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:42.959585 22868 raft_consensus.cc:374] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:42.959663 22868 raft_consensus.cc:729] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a35523d6fe4436eb10289f3045fdf9c, State: Initialized, Role: FOLLOWER
I20260808 10:43:42.960115 22868 consensus_queue.cc:260] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:42.960764 22850 heartbeater.cc:503] Master 127.1.194.126:45907 was elected leader, sending a full tablet report...
I20260808 10:43:42.960706 22871 log.cc:824] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:42.961114 22868 ts_tablet_manager.cc:1428] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 10:43:42.963074 22871 tablet_bootstrap.cc:492] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: No bootstrap required, opened a new log
I20260808 10:43:42.963276 22871 ts_tablet_manager.cc:1397] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:43:42.963898 22867 raft_consensus.cc:348] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:42.964197 22867 raft_consensus.cc:374] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:42.964303 22867 raft_consensus.cc:729] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d80dc5c238a04b22a80edd43c39aa188, State: Initialized, Role: FOLLOWER
I20260808 10:43:42.964761 22867 consensus_queue.cc:260] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:42.965513 22577 heartbeater.cc:503] Master 127.1.194.126:45907 was elected leader, sending a full tablet report...
I20260808 10:43:42.965866 22867 ts_tablet_manager.cc:1428] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 10:43:42.967253 22871 raft_consensus.cc:348] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:42.967535 22871 raft_consensus.cc:374] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:42.967602 22871 raft_consensus.cc:729] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d93fb7d29577467ab6781cd83c6f8db7, State: Initialized, Role: FOLLOWER
I20260808 10:43:42.968055 22871 consensus_queue.cc:260] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:42.968717 22714 heartbeater.cc:503] Master 127.1.194.126:45907 was elected leader, sending a full tablet report...
I20260808 10:43:42.969026 22871 ts_tablet_manager.cc:1428] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:43.089135 22872 raft_consensus.cc:482] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:43.089390 22872 raft_consensus.cc:504] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:43.090561 22872 leader_election.cc:288] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d80dc5c238a04b22a80edd43c39aa188 (127.1.194.65:42591), d93fb7d29577467ab6781cd83c6f8db7 (127.1.194.66:33637)
I20260808 10:43:43.095712 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.091061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:44157 (local address 127.1.194.66:33637)
0808 10:43:43.091192 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:43.091197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:43.091539 (+ 342us) server_negotiation.cc:407] Connection header received
0808 10:43:43.091655 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:43.091657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:43.091743 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:43.091830 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:43.092093 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.092803 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.094340 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.094988 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.095128 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.095178 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:43.095501 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 10:43:43.095576 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:43:43.095712 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.091074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:40681 (local address 127.1.194.65:42591)
0808 10:43:43.091197 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:43.091202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:43.091218 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:43:43.091282 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:43.091284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:43.091328 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:43.091401 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:43.091787 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.092637 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.094340 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.094988 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.095126 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.095178 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:43.095501 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 10:43:43.095592 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:43:43.095738 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.090901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:42591 (local address 127.1.194.67:40681)
0808 10:43:43.091013 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:43.091031 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:43.091135 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:43.091448 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:43.091454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:43.091480 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:43.091647 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.091654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.092759 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.092761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.094180 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.094192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.095148 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.095153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.095337 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.095382 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:43:43.095466 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:43:43.095578 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":147,"spinlock_wait_cycles":35456}
I20260808 10:43:43.095752 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.090901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:33637 (local address 127.1.194.67:44157)
0808 10:43:43.091417 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:43.091432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:43.091549 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:43.091876 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:43.091878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:43.091894 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:43.091982 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.091986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.092902 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.092906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.094183 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.094192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.095150 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.095153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.095334 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.095378 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:43:43.095466 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:43:43.095578 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":227,"spinlock_wait_cycles":384,"thread_start_us":131,"threads_started":1}
I20260808 10:43:43.096525 22530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac123a0b4c4a0cb29bfaca13add06c" candidate_uuid: "0a35523d6fe4436eb10289f3045fdf9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d80dc5c238a04b22a80edd43c39aa188" is_pre_election: true
I20260808 10:43:43.096570 22668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac123a0b4c4a0cb29bfaca13add06c" candidate_uuid: "0a35523d6fe4436eb10289f3045fdf9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93fb7d29577467ab6781cd83c6f8db7" is_pre_election: true
I20260808 10:43:43.096870 22668 raft_consensus.cc:2454] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a35523d6fe4436eb10289f3045fdf9c in term 0.
I20260808 10:43:43.096880 22530 raft_consensus.cc:2454] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a35523d6fe4436eb10289f3045fdf9c in term 0.
I20260808 10:43:43.097366 22738 leader_election.cc:302] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a35523d6fe4436eb10289f3045fdf9c, d93fb7d29577467ab6781cd83c6f8db7; no voters:
I20260808 10:43:43.097678 22872 raft_consensus.cc:2781] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:43:43.097769 22872 raft_consensus.cc:482] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:43:43.097810 22872 raft_consensus.cc:3037] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:43.098841 22872 raft_consensus.cc:504] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:43.099197 22872 leader_election.cc:288] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [CANDIDATE]: Term 1 election: Requested vote from peers d80dc5c238a04b22a80edd43c39aa188 (127.1.194.65:42591), d93fb7d29577467ab6781cd83c6f8db7 (127.1.194.66:33637)
I20260808 10:43:43.099503 22530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac123a0b4c4a0cb29bfaca13add06c" candidate_uuid: "0a35523d6fe4436eb10289f3045fdf9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d80dc5c238a04b22a80edd43c39aa188"
I20260808 10:43:43.099617 22668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac123a0b4c4a0cb29bfaca13add06c" candidate_uuid: "0a35523d6fe4436eb10289f3045fdf9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93fb7d29577467ab6781cd83c6f8db7"
I20260808 10:43:43.099642 22530 raft_consensus.cc:3037] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:43.099761 22668 raft_consensus.cc:3037] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:43.100807 22530 raft_consensus.cc:2454] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a35523d6fe4436eb10289f3045fdf9c in term 1.
I20260808 10:43:43.100811 22668 raft_consensus.cc:2454] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a35523d6fe4436eb10289f3045fdf9c in term 1.
I20260808 10:43:43.101091 22738 leader_election.cc:302] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a35523d6fe4436eb10289f3045fdf9c, d93fb7d29577467ab6781cd83c6f8db7; no voters:
I20260808 10:43:43.101316 22872 raft_consensus.cc:2781] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:43.101574 22872 raft_consensus.cc:686] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 1 LEADER]: Becoming Leader. State: Replica: 0a35523d6fe4436eb10289f3045fdf9c, State: Running, Role: LEADER
I20260808 10:43:43.101926 22872 consensus_queue.cc:237] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } }
I20260808 10:43:43.105229 22383 catalog_manager.cc:5495] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a35523d6fe4436eb10289f3045fdf9c (127.1.194.67). New cstate: current_term: 1 leader_uuid: "0a35523d6fe4436eb10289f3045fdf9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a35523d6fe4436eb10289f3045fdf9c" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 39135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 } health_report { overall_health: UNKNOWN } } }
I20260808 10:43:43.129042 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.125448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48710 (local address 127.1.194.67:39135)
0808 10:43:43.125611 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:43.125614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:43.125628 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:43:43.125729 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:43.125731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:43.125783 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:43.125868 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:43:43.126075 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.126566 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.127505 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.127724 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.127827 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.128072 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:43:43.128589 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:43:43.128705 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:43.128786 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:43:43.128839 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:43:43.134867 22668 raft_consensus.cc:1261] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 1 FOLLOWER]: Refusing update from remote peer 0a35523d6fe4436eb10289f3045fdf9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:43.134953 22530 raft_consensus.cc:1261] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 1 FOLLOWER]: Refusing update from remote peer 0a35523d6fe4436eb10289f3045fdf9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 10:43:43.135639 22851 tablet.cc:2369] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:43:43.135632 22877 consensus_queue.cc:1035] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d93fb7d29577467ab6781cd83c6f8db7" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 33637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:43.135847 22872 consensus_queue.cc:1035] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d80dc5c238a04b22a80edd43c39aa188" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 42591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:43.143566 22882 mvcc.cc:205] Tried to move back new op lower bound from 7316217131552743424 to 7316217131428102144. Current Snapshot: MvccSnapshot[applied={T|T < 7316217131552743424}]
I20260808 10:43:43.145754 22881 mvcc.cc:205] Tried to move back new op lower bound from 7316217131552743424 to 7316217131428102144. Current Snapshot: MvccSnapshot[applied={T|T < 7316217131552743424}]
I20260808 10:43:43.152957 22380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:43.153119 22380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:43.155804 22380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4162b239b577423ab1abdd952f0b6674: Sending DeleteTablet for 3 replicas of tablet f8ac123a0b4c4a0cb29bfaca13add06c
I20260808 10:43:43.156677 22647 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ac123a0b4c4a0cb29bfaca13add06c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:43 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:45320
I20260808 10:43:43.156678 22510 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ac123a0b4c4a0cb29bfaca13add06c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:43 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:51688
I20260808 10:43:43.157295 22892 tablet_replica.cc:330] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188: stopping tablet replica
I20260808 10:43:43.157692 22892 raft_consensus.cc:2229] T f8ac123a0b4c4a0cb29bfaca13add06c P d80dc5c238a04b22a80edd43c39aa188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:43.157675 22784 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ac123a0b4c4a0cb29bfaca13add06c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:43 UTC) from {username='kudu', principal='kudu/127.1.194.126@KRBTEST.COM'} at 127.0.0.1:48702
I20260808 10:43:43.158447 22893 tablet_replica.cc:330] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: stopping tablet replica
I20260808 10:43:43.158496 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22445
I20260808 10:43:43.158723 22893 raft_consensus.cc:2229] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:43:43.159065 22893 raft_consensus.cc:2258] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:43.162647 22893 ts_tablet_manager.cc:1905] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:43.165437 22893 ts_tablet_manager.cc:1918] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:43.165546 22893 log.cc:1196] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/f8ac123a0b4c4a0cb29bfaca13add06c
I20260808 10:43:43.165899 22893 ts_tablet_manager.cc:1939] T f8ac123a0b4c4a0cb29bfaca13add06c P 0a35523d6fe4436eb10289f3045fdf9c: Deleting consensus metadata
I20260808 10:43:43.167059 22369 catalog_manager.cc:4841] TS 0a35523d6fe4436eb10289f3045fdf9c (127.1.194.67:39135): tablet f8ac123a0b4c4a0cb29bfaca13add06c (table test-table [id=ee0222068ec64871bcfd80644ed61b4d]) successfully deleted
I20260808 10:43:43.168780 22891 tablet_replica.cc:330] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: stopping tablet replica
I20260808 10:43:43.169155 22891 raft_consensus.cc:2229] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:43.169543 22891 raft_consensus.cc:2258] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:43.174319 22891 ts_tablet_manager.cc:1905] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:43:43.175920 22366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:42591 (error 108)
I20260808 10:43:43.176823 22891 ts_tablet_manager.cc:1918] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:43.177034 22891 log.cc:1196] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/f8ac123a0b4c4a0cb29bfaca13add06c
I20260808 10:43:43.177047 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.176720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:42591 (local address 127.0.0.1:51690)
0808 10:43:43.176882 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:43.176958 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 10:43:43.177129 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22582
W20260808 10:43:43.177215 22366 catalog_manager.cc:4584] TS d80dc5c238a04b22a80edd43c39aa188 (127.1.194.65:42591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8ac123a0b4c4a0cb29bfaca13add06c: Network error: Client connection negotiation failed: client connection to 127.1.194.65:42591: connect: Connection refused (error 111)
I20260808 10:43:43.177497 22891 ts_tablet_manager.cc:1939] T f8ac123a0b4c4a0cb29bfaca13add06c P d93fb7d29577467ab6781cd83c6f8db7: Deleting consensus metadata
I20260808 10:43:43.184149 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22719
I20260808 10:43:43.185365 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.184360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:33637 (local address 127.0.0.1:45336)
0808 10:43:43.185240 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:43.185292 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.66:33637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":805}
I20260808 10:43:43.191043 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22350
2026-08-08T10:43:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2451 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T10:43:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:43Z Disabled control of system clock
I20260808 10:43:43.223802 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:39617
--webserver_interface=127.1.194.126
--webserver_port=0
--builtin_ntp_servers=127.1.194.84:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:39617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:43:43.332798 22898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:43.333053 22898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:43.333099 22898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:43.336990 22898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:43.337047 22898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:43.337064 22898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:43.337085 22898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:43.337100 22898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:43.341871 22898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:39617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:39617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:43.342902 22898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:43.344197 22898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:43.352828 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:43.352936 22898 server_base.cc:1034] running on GCE node
W20260808 10:43:43.352818 22903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:43.352845 22904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:43.353713 22898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:43.354861 22898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:43.356086 22898 hybrid_clock.cc:648] HybridClock initialized: now 1786185823356070 us; error 28 us; skew 500 ppm
I20260808 10:43:43.358134 22898 webserver.cc:460] Webserver started at http://127.1.194.126:35653/ using document root <none> and password file <none>
I20260808 10:43:43.358757 22898 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:43.358860 22898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:43.359102 22898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:43.361255 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "dbc41e6b4ccc4059a1e232d1d4b28900"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "bf3ec8f46a4869d51d2909df8a5dd36e"
server_key_iv: "fd6998566c8f7050b0b66b3a8475a42e"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.361886 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "dbc41e6b4ccc4059a1e232d1d4b28900"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "bf3ec8f46a4869d51d2909df8a5dd36e"
server_key_iv: "fd6998566c8f7050b0b66b3a8475a42e"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.365733 22898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:43:43.368144 22913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.369392 22898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:43.369562 22898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "dbc41e6b4ccc4059a1e232d1d4b28900"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "bf3ec8f46a4869d51d2909df8a5dd36e"
server_key_iv: "fd6998566c8f7050b0b66b3a8475a42e"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.369695 22898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:43.383277 22898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:43.383975 22898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:43.384182 22898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:43.390967 22898 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:39617
I20260808 10:43:43.390977 22966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:39617 every 8 connection(s)
I20260808 10:43:43.391893 22898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:43.394969 22967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:43.400012 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22898
I20260808 10:43:43.400182 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:43.400226 22967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900: Bootstrap starting.
I20260808 10:43:43.400501 1801 external_mini_cluster.cc:1428] Setting key 9514e2de406243ff370323f5a077f944
I20260808 10:43:43.402789 22967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:43.403517 22967 log.cc:824] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:43.405272 22967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900: No bootstrap required, opened a new log
I20260808 10:43:43.408620 22967 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39617 } }
I20260808 10:43:43.408879 22967 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:43.408991 22967 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc41e6b4ccc4059a1e232d1d4b28900, State: Initialized, Role: FOLLOWER
I20260808 10:43:43.409420 22967 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39617 } }
I20260808 10:43:43.409613 22967 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:43.409687 22967 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:43.409837 22967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:43.409916 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.401874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.1.194.126:39617)
0808 10:43:43.402596 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:43.402612 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:43.402649 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:43:43.403334 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:43.403351 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:43.403786 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:43.404111 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:43.404521 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.405575 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.406469 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.406940 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.407111 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.407965 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:43.407971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:43.407991 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:43.408360 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:43.408416 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:43.408444 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:43.408555 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:43.408669 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 10:43:43.408933 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":136,"threads_started":1}
I20260808 10:43:43.410787 22967 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39617 } }
I20260808 10:43:43.411214 22967 leader_election.cc:302] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbc41e6b4ccc4059a1e232d1d4b28900; no voters:
I20260808 10:43:43.411564 22967 leader_election.cc:288] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:43.411682 22972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:43.411917 22972 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [term 1 LEADER]: Becoming Leader. State: Replica: dbc41e6b4ccc4059a1e232d1d4b28900, State: Running, Role: LEADER
I20260808 10:43:43.412266 22972 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39617 } }
I20260808 10:43:43.412842 22967 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:43.414168 22973 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39617 } } }
I20260808 10:43:43.414307 22973 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:43.414553 22974 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc41e6b4ccc4059a1e232d1d4b28900. Latest consensus state: current_term: 1 leader_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc41e6b4ccc4059a1e232d1d4b28900" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 39617 } } }
I20260808 10:43:43.414743 22974 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:43.415531 22981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:43.418576 22981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:43.423826 22981 catalog_manager.cc:1347] Generated new cluster ID: af94504ea23642b19d62ce2d4d749acf
I20260808 10:43:43.423904 22981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:43.451186 22981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:43.452457 22981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:43.460232 22981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900: Generated new TSK 0
I20260808 10:43:43.461035 22981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:43.472012 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.65:0
--local_ip_for_outbound_sockets=127.1.194.65
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39617
--builtin_ntp_servers=127.1.194.84:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:43:43.583763 22991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:43.584066 22991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:43.584153 22991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:43.588258 22991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:43.588358 22991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:43.588444 22991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.65
I20260808 10:43:43.593242 22991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.194.65
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.65
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:43.594399 22991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:43.595801 22991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:43.605690 22996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:43.605690 22997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:43.605703 22999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:43.605744 22991 server_base.cc:1034] running on GCE node
I20260808 10:43:43.606366 22991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:43.607139 22991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:43.608354 22991 hybrid_clock.cc:648] HybridClock initialized: now 1786185823608322 us; error 50 us; skew 500 ppm
I20260808 10:43:43.610387 22991 webserver.cc:460] Webserver started at http://127.1.194.65:40981/ using document root <none> and password file <none>
I20260808 10:43:43.611038 22991 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:43.611140 22991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:43.611389 22991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:43.613794 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data/instance:
uuid: "c34dc5469050484f81150a44288eec9e"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "31175cc9fdeeb5559b6b8984d414e384"
server_key_iv: "0a40845172fa385c7c93951c4f5e0fa6"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.614492 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance:
uuid: "c34dc5469050484f81150a44288eec9e"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "31175cc9fdeeb5559b6b8984d414e384"
server_key_iv: "0a40845172fa385c7c93951c4f5e0fa6"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.618433 22991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:43:43.621013 23006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.622800 22991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:43.622938 22991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal
uuid: "c34dc5469050484f81150a44288eec9e"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "31175cc9fdeeb5559b6b8984d414e384"
server_key_iv: "0a40845172fa385c7c93951c4f5e0fa6"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.623085 22991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:43.639599 22991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:43.640268 22991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:43.640465 22991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:43.641028 22991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:43.642016 22991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:43.642064 22991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.642141 22991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:43.642192 22991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.650760 22991 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.65:37699
I20260808 10:43:43.650784 23121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.65:37699 every 8 connection(s)
I20260808 10:43:43.651656 22991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/data/info.pb
I20260808 10:43:43.659519 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 22991
I20260808 10:43:43.659698 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-0/wal/instance
I20260808 10:43:43.659978 1801 external_mini_cluster.cc:1428] Setting key 1b3d76e3d7c49f7fb141a3aefe3ec9ae
I20260808 10:43:43.661058 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.653754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.65:56065 (local address 127.1.194.126:39617)
0808 10:43:43.653895 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:43.653898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:43.654725 (+ 827us) server_negotiation.cc:407] Connection header received
0808 10:43:43.656201 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:43.656204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:43.656263 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:43.656371 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:43.657231 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.657854 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.658972 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.659214 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.659338 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.660475 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:43.660479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:43.660481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:43.660493 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:43.660533 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:43.660548 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:43.660641 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:43.660859 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 10:43:43.660921 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:43:43.662022 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.66:0
--local_ip_for_outbound_sockets=127.1.194.66
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39617
--builtin_ntp_servers=127.1.194.84:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:43:43.662027 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.653943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39617 (local address 127.1.194.65:56065)
0808 10:43:43.654563 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:43.654605 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:43.655913 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:43.656532 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:43.656538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:43.656623 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:43.657089 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.657102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.657981 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.657989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.658841 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.658850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.659376 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.659381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.659539 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.660194 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:43.660196 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:43.660233 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:43.660294 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:43.660658 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:43.660667 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:43.660673 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:43:43.660835 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 10:43:43.661100 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
I20260808 10:43:43.663545 23122 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39617
I20260808 10:43:43.663805 23122 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:43.664341 23122 heartbeater.cc:511] Master 127.1.194.126:39617 requested a full tablet report, sending...
I20260808 10:43:43.666019 22930 ts_manager.cc:194] Registered new tserver with Master: c34dc5469050484f81150a44288eec9e (127.1.194.65:37699)
I20260808 10:43:43.667302 22930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.65:56065
W20260808 10:43:43.776206 23125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:43.776458 23125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:43.776494 23125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:43.780592 23125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:43.780654 23125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:43.780709 23125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.66
I20260808 10:43:43.785420 23125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.194.66
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.66
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:43.786538 23125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:43.788062 23125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:43.798048 23133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:43.798038 23131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:43.798029 23125 server_base.cc:1034] running on GCE node
W20260808 10:43:43.798010 23130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:43.798689 23125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:43.799387 23125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:43.800684 23125 hybrid_clock.cc:648] HybridClock initialized: now 1786185823800666 us; error 28 us; skew 500 ppm
I20260808 10:43:43.802896 23125 webserver.cc:460] Webserver started at http://127.1.194.66:45917/ using document root <none> and password file <none>
I20260808 10:43:43.803547 23125 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:43.803653 23125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:43.803906 23125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:43.806344 23125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data/instance:
uuid: "6272d09d5bbf44eea51358bcb6ea3d2a"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "6e913f29de6662e9750e8c612265fec9"
server_key_iv: "c20dfbbc85359ff79901b3148eb77f34"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.806973 23125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance:
uuid: "6272d09d5bbf44eea51358bcb6ea3d2a"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "6e913f29de6662e9750e8c612265fec9"
server_key_iv: "c20dfbbc85359ff79901b3148eb77f34"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.811077 23125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:43:43.813649 23139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.815038 23125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:43:43.815167 23125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal
uuid: "6272d09d5bbf44eea51358bcb6ea3d2a"
format_stamp: "Formatted at 2026-08-08 10:43:43 on dist-test-slave-ss4n"
server_key: "6e913f29de6662e9750e8c612265fec9"
server_key_iv: "c20dfbbc85359ff79901b3148eb77f34"
server_key_version: "encryptionkey@0"
I20260808 10:43:43.815289 23125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:43.826948 23125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:43.827751 23125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:43.827946 23125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:43.828542 23125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:43.829578 23125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:43.829629 23125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.829669 23125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:43.829685 23125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:43.838117 23125 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.66:34835
I20260808 10:43:43.838186 23254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.66:34835 every 8 connection(s)
I20260808 10:43:43.839036 23125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/data/info.pb
I20260808 10:43:43.839529 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23125
I20260808 10:43:43.839654 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal/instance
I20260808 10:43:43.839989 1801 external_mini_cluster.cc:1428] Setting key 44bb1503f44c48c35f24a64b084fd4e3
I20260808 10:43:43.842293 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.194.67:0
--local_ip_for_outbound_sockets=127.1.194.67
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39617
--builtin_ntp_servers=127.1.194.84:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:43:43.851613 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.840999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.66:47651 (local address 127.1.194.126:39617)
0808 10:43:43.841149 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:43.841153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:43.842594 (+ 1441us) server_negotiation.cc:407] Connection header received
0808 10:43:43.844607 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:43.844613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:43.844675 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:43.844779 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:43.845803 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.846735 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.848679 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.849113 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.849214 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.850905 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:43.850912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:43.850917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:43.850936 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:43.850963 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:43.850976 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:43.851045 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:43.851318 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 10:43:43.851399 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:43:43.853557 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:43.841616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39617 (local address 127.1.194.66:47651)
0808 10:43:43.842418 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:43.842479 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:43.844388 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:43.844934 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:43.844941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:43.845023 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:43.845647 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.845663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.846902 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.846908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.848514 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:43.848528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:43.849451 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:43.849457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:43.849741 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:43.850535 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:43.850539 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:43.850603 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:43.850703 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:43.851127 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:43.851139 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:43.851146 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:43:43.851940 (+ 794us) client_negotiation.cc:240] Negotiation successful
0808 10:43:43.852317 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260808 10:43:43.856144 23255 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39617
I20260808 10:43:43.856426 23255 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:43.857108 23255 heartbeater.cc:511] Master 127.1.194.126:39617 requested a full tablet report, sending...
I20260808 10:43:43.858319 22930 ts_manager.cc:194] Registered new tserver with Master: 6272d09d5bbf44eea51358bcb6ea3d2a (127.1.194.66:34835)
I20260808 10:43:43.859158 22930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.66:47651
W20260808 10:43:43.974434 23258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:43.974735 23258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:43.974807 23258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:43.978816 23258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:43.978896 23258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:43.978977 23258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.194.67
I20260808 10:43:43.984673 23258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.194.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.194.67
--webserver_port=0
--tserver_master_addrs=127.1.194.126:39617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.194.67
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:43.986011 23258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:43.987553 23258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:43.997649 23263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:43.997689 23266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:43.997784 23264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:43.998000 23258 server_base.cc:1034] running on GCE node
I20260808 10:43:43.998409 23258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:43.999094 23258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:44.000289 23258 hybrid_clock.cc:648] HybridClock initialized: now 1786185824000262 us; error 46 us; skew 500 ppm
I20260808 10:43:44.005157 23258 webserver.cc:460] Webserver started at http://127.1.194.67:44405/ using document root <none> and password file <none>
I20260808 10:43:44.005834 23258 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:44.005931 23258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:44.006191 23258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:44.008325 23258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data/instance:
uuid: "9bcb38fe061e4b80b3d3e4618bd57c33"
format_stamp: "Formatted at 2026-08-08 10:43:44 on dist-test-slave-ss4n"
server_key: "6045306f5b087a87578627f313a1f58f"
server_key_iv: "fdcca1aa2f8ae8ccbdb64c398e0b49b6"
server_key_version: "encryptionkey@0"
I20260808 10:43:44.008898 23258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance:
uuid: "9bcb38fe061e4b80b3d3e4618bd57c33"
format_stamp: "Formatted at 2026-08-08 10:43:44 on dist-test-slave-ss4n"
server_key: "6045306f5b087a87578627f313a1f58f"
server_key_iv: "fdcca1aa2f8ae8ccbdb64c398e0b49b6"
server_key_version: "encryptionkey@0"
I20260808 10:43:44.012801 23258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 10:43:44.015661 23272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:44.017045 23258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:43:44.017195 23258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal
uuid: "9bcb38fe061e4b80b3d3e4618bd57c33"
format_stamp: "Formatted at 2026-08-08 10:43:44 on dist-test-slave-ss4n"
server_key: "6045306f5b087a87578627f313a1f58f"
server_key_iv: "fdcca1aa2f8ae8ccbdb64c398e0b49b6"
server_key_version: "encryptionkey@0"
I20260808 10:43:44.017323 23258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:44.029735 23258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:44.030417 23258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:44.030622 23258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:44.031198 23258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:43:44.032135 23258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:43:44.032210 23258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:44.032279 23258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:43:44.032346 23258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:44.041155 23258 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.67:41497
I20260808 10:43:44.041178 23387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.67:41497 every 8 connection(s)
I20260808 10:43:44.042112 23258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/data/info.pb
I20260808 10:43:44.050386 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23258
I20260808 10:43:44.050493 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal/instance
I20260808 10:43:44.050815 1801 external_mini_cluster.cc:1428] Setting key 4a6f1a45712250ad7dac0dd9398bdfa5
I20260808 10:43:44.052160 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.043959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:33459 (local address 127.1.194.126:39617)
0808 10:43:44.044115 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.044119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.045107 (+ 988us) server_negotiation.cc:407] Connection header received
0808 10:43:44.046567 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.046571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.046625 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.046736 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:44.047624 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.048225 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.049793 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.050025 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.050192 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.051597 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:44.051601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:44.051604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:44.051618 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:44.051640 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:44.051651 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:44.051732 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.051947 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.052014 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:43:44.053186 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.044297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:39617 (local address 127.1.194.67:33459)
0808 10:43:44.044927 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.044967 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:44.046292 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:44.046897 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:44.046903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:44.046983 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:43:44.047462 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.047475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.048419 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.048423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.049399 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.049409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.050219 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.050222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.050369 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.051119 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:43:44.051120 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:43:44.051162 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:43:44.051229 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:43:44.051796 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:43:44.051804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:43:44.051809 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:43:44.051946 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 10:43:44.052271 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260808 10:43:44.054301 23388 heartbeater.cc:347] Connected to a master server at 127.1.194.126:39617
I20260808 10:43:44.054556 23388 heartbeater.cc:464] Registering TS with master...
I20260808 10:43:44.055094 23388 heartbeater.cc:511] Master 127.1.194.126:39617 requested a full tablet report, sending...
I20260808 10:43:44.056105 22930 ts_manager.cc:194] Registered new tserver with Master: 9bcb38fe061e4b80b3d3e4618bd57c33 (127.1.194.67:41497)
I20260808 10:43:44.056700 22930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.194.67:33459
I20260808 10:43:44.065035 1801 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:43:44.071237 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.067142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50328 (local address 127.1.194.126:39617)
0808 10:43:44.067303 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.067308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.067464 (+ 156us) server_negotiation.cc:407] Connection header received
0808 10:43:44.067670 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.067673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.067728 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.067826 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:44.068147 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.068674 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.069608 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.069843 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.069956 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.070806 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:44.070810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:44.070813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:44.070825 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:44.070844 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:44.070853 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:44.070939 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.071029 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.071092 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:43:44.074545 22930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:43:44.077551 22930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:43:44.092378 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.086226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.1.194.65:37699)
0808 10:43:44.086719 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.086732 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.087144 (+ 412us) server_negotiation.cc:407] Connection header received
0808 10:43:44.087442 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.087450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.087685 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.087808 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:44.088360 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.089157 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.090898 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.091666 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.091796 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.091999 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.092133 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.092231 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":108,"threads_started":1}
I20260808 10:43:44.092377 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.086113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37699 (local address 127.0.0.1:56970)
0808 10:43:44.086995 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.087049 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:44.087250 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:44.088002 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:44.088008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:44.088032 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:44.088234 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.088242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.089283 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.089288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.090746 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.090757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.091798 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.091802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.091905 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.091941 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:43:44.092069 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 10:43:44.092203 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":30,"spinlock_wait_cycles":8960,"thread_start_us":76,"threads_started":1}
I20260808 10:43:44.092753 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.086919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.1.194.66:34835)
0808 10:43:44.087511 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.087525 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.087549 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:43:44.087631 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.087639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.087832 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.087982 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:44.088419 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.089669 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.091032 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.091719 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.091825 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.091997 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.092465 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.092565 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":113,"threads_started":1}
I20260808 10:43:44.092847 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.086174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:34835 (local address 127.0.0.1:49842)
0808 10:43:44.087374 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.087390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:44.087482 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:44.088002 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:44.088008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:44.088047 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:44.088236 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.088242 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.089791 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.089795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.090912 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.090919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.092078 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.092082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.092176 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.092200 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:43:44.092274 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:43:44.092343 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"mutex_wait_us":64,"spinlock_wait_cycles":19200,"thread_start_us":117,"threads_started":1}
I20260808 10:43:44.093204 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.086922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.1.194.67:41497)
0808 10:43:44.087327 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.087338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.087727 (+ 389us) server_negotiation.cc:407] Connection header received
0808 10:43:44.087805 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.087810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.087971 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.088108 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:44.088449 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.090001 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.091417 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.092194 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.092296 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.092466 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.092921 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.093020 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":160,"threads_started":1}
I20260808 10:43:44.093696 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.086454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.67:41497 (local address 127.0.0.1:54810)
0808 10:43:44.087605 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.087619 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:44.087713 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:44.088156 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:44.088159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:44.088176 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:44.088323 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.088330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.090136 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.090139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.091278 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.091289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.092528 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.092531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.092612 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.092634 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:43:44.092710 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:43:44.092781 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260808 10:43:44.095600 23187 tablet_service.cc:1467] Processing CreateTablet for tablet 4db0c104d9f64b1ea8118d23109e898f (DEFAULT_TABLE table=test-table [id=0f5a1560782944b8b668b05a5093de0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:44.095676 23055 tablet_service.cc:1467] Processing CreateTablet for tablet 4db0c104d9f64b1ea8118d23109e898f (DEFAULT_TABLE table=test-table [id=0f5a1560782944b8b668b05a5093de0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:44.096630 23187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db0c104d9f64b1ea8118d23109e898f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:44.096702 23055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db0c104d9f64b1ea8118d23109e898f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:44.096338 23321 tablet_service.cc:1467] Processing CreateTablet for tablet 4db0c104d9f64b1ea8118d23109e898f (DEFAULT_TABLE table=test-table [id=0f5a1560782944b8b668b05a5093de0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:43:44.097265 23321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db0c104d9f64b1ea8118d23109e898f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:44.103075 23405 tablet_bootstrap.cc:492] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Bootstrap starting.
I20260808 10:43:44.104449 23406 tablet_bootstrap.cc:492] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: Bootstrap starting.
I20260808 10:43:44.104815 23407 tablet_bootstrap.cc:492] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Bootstrap starting.
I20260808 10:43:44.105333 23405 tablet_bootstrap.cc:654] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:44.106087 23405 log.cc:824] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:44.106642 23406 tablet_bootstrap.cc:654] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:44.106642 23407 tablet_bootstrap.cc:654] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:44.107470 23407 log.cc:824] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:44.107470 23406 log.cc:824] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:44.107952 23405 tablet_bootstrap.cc:492] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: No bootstrap required, opened a new log
I20260808 10:43:44.108150 23405 ts_tablet_manager.cc:1397] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:43:44.109311 23407 tablet_bootstrap.cc:492] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: No bootstrap required, opened a new log
I20260808 10:43:44.109314 23406 tablet_bootstrap.cc:492] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: No bootstrap required, opened a new log
I20260808 10:43:44.109513 23406 ts_tablet_manager.cc:1397] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:43:44.109513 23407 ts_tablet_manager.cc:1397] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260808 10:43:44.111476 23405 raft_consensus.cc:348] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.111725 23405 raft_consensus.cc:374] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:44.111797 23405 raft_consensus.cc:729] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6272d09d5bbf44eea51358bcb6ea3d2a, State: Initialized, Role: FOLLOWER
I20260808 10:43:44.112265 23405 consensus_queue.cc:260] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.113003 23255 heartbeater.cc:503] Master 127.1.194.126:39617 was elected leader, sending a full tablet report...
I20260808 10:43:44.113323 23405 ts_tablet_manager.cc:1428] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:43:44.113881 23406 raft_consensus.cc:348] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.114184 23406 raft_consensus.cc:374] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:44.114250 23406 raft_consensus.cc:729] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34dc5469050484f81150a44288eec9e, State: Initialized, Role: FOLLOWER
I20260808 10:43:44.114493 23407 raft_consensus.cc:348] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.114728 23406 consensus_queue.cc:260] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.114804 23407 raft_consensus.cc:374] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:44.114868 23407 raft_consensus.cc:729] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcb38fe061e4b80b3d3e4618bd57c33, State: Initialized, Role: FOLLOWER
I20260808 10:43:44.115294 23407 consensus_queue.cc:260] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.115398 23122 heartbeater.cc:503] Master 127.1.194.126:39617 was elected leader, sending a full tablet report...
I20260808 10:43:44.115810 23406 ts_tablet_manager.cc:1428] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:43:44.116300 23407 ts_tablet_manager.cc:1428] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:43:44.116350 23388 heartbeater.cc:503] Master 127.1.194.126:39617 was elected leader, sending a full tablet report...
W20260808 10:43:44.152835 23123 tablet.cc:2369] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:43:44.286171 23413 raft_consensus.cc:482] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:43:44.286441 23413 raft_consensus.cc:504] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.287762 23413 leader_election.cc:288] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c34dc5469050484f81150a44288eec9e (127.1.194.65:37699), 6272d09d5bbf44eea51358bcb6ea3d2a (127.1.194.66:34835)
I20260808 10:43:44.292086 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.288055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37699 (local address 127.1.194.67:55889)
0808 10:43:44.288190 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.288208 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:44.288308 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:44.288599 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:44.288601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:44.288623 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:44.288837 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.288847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.289879 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.289883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.290784 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.290794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.291662 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.291665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.291767 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.291794 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:43:44.291870 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:43:44.291941 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"spinlock_wait_cycles":27008}
I20260808 10:43:44.292160 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.288218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:55889 (local address 127.1.194.65:37699)
0808 10:43:44.288345 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.288351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.288368 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:43:44.288428 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.288431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.288479 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.288562 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:44.289039 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.289734 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.290937 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.291501 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.291604 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.291645 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.291915 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.292014 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:43:44.292423 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.288315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.194.67:36729 (local address 127.1.194.66:34835)
0808 10:43:44.288471 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.288475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.288703 (+ 228us) server_negotiation.cc:407] Connection header received
0808 10:43:44.288836 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.288839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.288903 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.289022 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:43:44.289355 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.289921 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.291303 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.291784 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.291906 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.291947 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.292202 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.292295 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:43:44.292419 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.288056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.66:34835 (local address 127.1.194.67:36729)
0808 10:43:44.288590 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.288605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:43:44.288721 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:43:44.289038 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:43:44.289040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:43:44.289058 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:43:44.289177 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.289183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.290102 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.290105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.291162 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:43:44.291173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.291935 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.291939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:43:44.292050 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.292074 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:43:44.292158 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:43:44.292236 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":75,"spinlock_wait_cycles":896,"thread_start_us":146,"threads_started":1}
W20260808 10:43:44.293100 23389 tablet.cc:2369] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:43:44.293018 23075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db0c104d9f64b1ea8118d23109e898f" candidate_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34dc5469050484f81150a44288eec9e" is_pre_election: true
I20260808 10:43:44.293082 23207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db0c104d9f64b1ea8118d23109e898f" candidate_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" is_pre_election: true
I20260808 10:43:44.293385 23075 raft_consensus.cc:2454] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bcb38fe061e4b80b3d3e4618bd57c33 in term 0.
I20260808 10:43:44.293385 23207 raft_consensus.cc:2454] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bcb38fe061e4b80b3d3e4618bd57c33 in term 0.
I20260808 10:43:44.294013 23273 leader_election.cc:302] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6272d09d5bbf44eea51358bcb6ea3d2a, 9bcb38fe061e4b80b3d3e4618bd57c33, c34dc5469050484f81150a44288eec9e; no voters:
I20260808 10:43:44.294328 23413 raft_consensus.cc:2781] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:43:44.294420 23413 raft_consensus.cc:482] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:43:44.294478 23413 raft_consensus.cc:3037] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:44.295511 23413 raft_consensus.cc:504] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.295915 23413 leader_election.cc:288] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [CANDIDATE]: Term 1 election: Requested vote from peers c34dc5469050484f81150a44288eec9e (127.1.194.65:37699), 6272d09d5bbf44eea51358bcb6ea3d2a (127.1.194.66:34835)
I20260808 10:43:44.296341 23207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db0c104d9f64b1ea8118d23109e898f" candidate_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a"
I20260808 10:43:44.296389 23075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db0c104d9f64b1ea8118d23109e898f" candidate_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34dc5469050484f81150a44288eec9e"
I20260808 10:43:44.296478 23207 raft_consensus.cc:3037] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:44.296546 23075 raft_consensus.cc:3037] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:44.297735 23075 raft_consensus.cc:2454] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bcb38fe061e4b80b3d3e4618bd57c33 in term 1.
I20260808 10:43:44.297897 23207 raft_consensus.cc:2454] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bcb38fe061e4b80b3d3e4618bd57c33 in term 1.
I20260808 10:43:44.298134 23276 leader_election.cc:302] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bcb38fe061e4b80b3d3e4618bd57c33, c34dc5469050484f81150a44288eec9e; no voters:
I20260808 10:43:44.298369 23413 raft_consensus.cc:2781] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:44.298635 23413 raft_consensus.cc:686] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 1 LEADER]: Becoming Leader. State: Replica: 9bcb38fe061e4b80b3d3e4618bd57c33, State: Running, Role: LEADER
I20260808 10:43:44.298966 23413 consensus_queue.cc:237] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } }
I20260808 10:43:44.302239 22930 catalog_manager.cc:5495] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bcb38fe061e4b80b3d3e4618bd57c33 (127.1.194.67). New cstate: current_term: 1 leader_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bcb38fe061e4b80b3d3e4618bd57c33" member_type: VOTER last_known_addr { host: "127.1.194.67" port: 41497 } health_report { overall_health: HEALTHY } } }
I20260808 10:43:44.329944 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.326186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54820 (local address 127.1.194.67:41497)
0808 10:43:44.326344 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:44.326349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:44.326364 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:43:44.326534 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:44.326537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:44.326610 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:44.326729 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:43:44.326970 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.327498 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.328429 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:44.328661 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:44.328772 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:44.329086 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:43:44.329549 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:43:44.329702 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:44.329743 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 10:43:44.329797 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:43:44.335907 23075 raft_consensus.cc:1261] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e [term 1 FOLLOWER]: Refusing update from remote peer 9bcb38fe061e4b80b3d3e4618bd57c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:44.335907 23207 raft_consensus.cc:1261] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 1 FOLLOWER]: Refusing update from remote peer 9bcb38fe061e4b80b3d3e4618bd57c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:43:44.336704 23413 consensus_queue.cc:1035] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34dc5469050484f81150a44288eec9e" member_type: VOTER last_known_addr { host: "127.1.194.65" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:43:44.336922 23415 consensus_queue.cc:1035] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6272d09d5bbf44eea51358bcb6ea3d2a" member_type: VOTER last_known_addr { host: "127.1.194.66" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:43:44.340260 23256 tablet.cc:2369] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:43:44.355062 22930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:44.355223 22930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:43:44.357792 22930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbc41e6b4ccc4059a1e232d1d4b28900: Sending DeleteTablet for 3 replicas of tablet 4db0c104d9f64b1ea8118d23109e898f
I20260808 10:43:44.358726 23187 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db0c104d9f64b1ea8118d23109e898f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:44 UTC) from {username='slave'} at 127.0.0.1:49842
I20260808 10:43:44.358773 23321 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db0c104d9f64b1ea8118d23109e898f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:44 UTC) from {username='slave'} at 127.0.0.1:54810
I20260808 10:43:44.358999 23055 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db0c104d9f64b1ea8118d23109e898f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:43:44 UTC) from {username='slave'} at 127.0.0.1:56970
I20260808 10:43:44.359288 23428 tablet_replica.cc:330] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: stopping tablet replica
I20260808 10:43:44.359627 23428 raft_consensus.cc:2229] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:43:44.359846 23430 tablet_replica.cc:330] T 4db0c104d9f64b1ea8118d23109e898f P c34dc5469050484f81150a44288eec9e: stopping tablet replica
I20260808 10:43:44.359969 23428 raft_consensus.cc:2258] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:44.360399 23429 tablet_replica.cc:330] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: stopping tablet replica
I20260808 10:43:44.360529 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22991
I20260808 10:43:44.360769 23429 raft_consensus.cc:2229] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:43:44.361174 23429 raft_consensus.cc:2258] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:43:44.361907 23428 ts_tablet_manager.cc:1905] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:44.362919 23429 ts_tablet_manager.cc:1905] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:43:44.365020 23428 ts_tablet_manager.cc:1918] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:44.365118 23428 log.cc:1196] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-1/wal/wals/4db0c104d9f64b1ea8118d23109e898f
I20260808 10:43:44.365454 23428 ts_tablet_manager.cc:1939] T 4db0c104d9f64b1ea8118d23109e898f P 6272d09d5bbf44eea51358bcb6ea3d2a: Deleting consensus metadata
I20260808 10:43:44.366155 23429 ts_tablet_manager.cc:1918] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:43:44.366279 23429 log.cc:1196] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Deleting WAL directory at /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185564023815-1801-0/minicluster-data/ts-2/wal/wals/4db0c104d9f64b1ea8118d23109e898f
I20260808 10:43:44.366647 23429 ts_tablet_manager.cc:1939] T 4db0c104d9f64b1ea8118d23109e898f P 9bcb38fe061e4b80b3d3e4618bd57c33: Deleting consensus metadata
I20260808 10:43:44.366940 22914 catalog_manager.cc:4841] TS 6272d09d5bbf44eea51358bcb6ea3d2a (127.1.194.66:34835): tablet 4db0c104d9f64b1ea8118d23109e898f (table test-table [id=0f5a1560782944b8b668b05a5093de0d]) successfully deleted
I20260808 10:43:44.367790 22915 catalog_manager.cc:4841] TS 9bcb38fe061e4b80b3d3e4618bd57c33 (127.1.194.67:41497): tablet 4db0c104d9f64b1ea8118d23109e898f (table test-table [id=0f5a1560782944b8b668b05a5093de0d]) successfully deleted
W20260808 10:43:44.373025 22917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.194.65:37699 (error 108)
I20260808 10:43:44.373282 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23125
I20260808 10:43:44.376055 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:44.375757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.65:37699 (local address 127.0.0.1:56984)
0808 10:43:44.375906 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:43:44.375980 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 10:43:44.376271 22917 catalog_manager.cc:4584] TS c34dc5469050484f81150a44288eec9e (127.1.194.65:37699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4db0c104d9f64b1ea8118d23109e898f: Network error: Client connection negotiation failed: client connection to 127.1.194.65:37699: connect: Connection refused (error 111)
I20260808 10:43:44.381313 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23258
I20260808 10:43:44.388149 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 22898
2026-08-08T10:43:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1196 ms)
[----------] 6 tests from EncryptionPolicyTest (10752 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:44 dist-test-slave-ss4n krb5kdc[23434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:44 dist-test-slave-ss4n krb5kdc[23434](info): set up 2 sockets
Aug 08 10:43:44 dist-test-slave-ss4n krb5kdc[23434](info): commencing operation
krb5kdc: starting...
W20260808 10:43:45.480824 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:45 dist-test-slave-ss4n krb5kdc[23434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:45.625291 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:45.746491 23447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:45.746843 23447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:45.746949 23447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:45.751219 23447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:45.751333 23447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:45.751379 23447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:45.751451 23447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:45.751495 23447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:45.756914 23447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:45.758112 23447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:45.759279 23447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:45.768698 23455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:45.768692 23452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:45.768723 23453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:45.769136 23447 server_base.cc:1034] running on GCE node
I20260808 10:43:45.769819 23447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:45.771063 23447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:45.772257 23447 hybrid_clock.cc:648] HybridClock initialized: now 1786185825772223 us; error 40 us; skew 500 ppm
Aug 08 10:43:45 dist-test-slave-ss4n krb5kdc[23434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185825, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:45.775437 23447 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:45.776376 23447 webserver.cc:460] Webserver started at http://127.0.0.1:46647/ using document root <none> and password file <none>
I20260808 10:43:45.776950 23447 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:45.777021 23447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:45.777280 23447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:45.779414 23447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "a26dd6aff9804dbaaed30dd43769ea15"
format_stamp: "Formatted at 2026-08-08 10:43:45 on dist-test-slave-ss4n"
server_key: "42b9ed84c2282385f073f0f1fff2da60"
server_key_iv: "c304278111b68c8bf88583634e2ccca3"
server_key_version: "encryptionkey@0"
I20260808 10:43:45.779976 23447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "a26dd6aff9804dbaaed30dd43769ea15"
format_stamp: "Formatted at 2026-08-08 10:43:45 on dist-test-slave-ss4n"
server_key: "42b9ed84c2282385f073f0f1fff2da60"
server_key_iv: "c304278111b68c8bf88583634e2ccca3"
server_key_version: "encryptionkey@0"
I20260808 10:43:45.783736 23447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:43:45.786221 23462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:45.787447 23447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:45.787590 23447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "a26dd6aff9804dbaaed30dd43769ea15"
format_stamp: "Formatted at 2026-08-08 10:43:45 on dist-test-slave-ss4n"
server_key: "42b9ed84c2282385f073f0f1fff2da60"
server_key_iv: "c304278111b68c8bf88583634e2ccca3"
server_key_version: "encryptionkey@0"
I20260808 10:43:45.787705 23447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:45.801744 23447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:45.805197 23447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:45.805359 23447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:45.812294 23447 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38267
I20260808 10:43:45.812386 23516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38267 every 8 connection(s)
I20260808 10:43:45.813290 23447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:45.816087 23517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:45.821383 23517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15: Bootstrap starting.
I20260808 10:43:45.823158 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23447
I20260808 10:43:45.823295 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:45.823591 1801 external_mini_cluster.cc:1428] Setting key 6893c7aee80209afda59dadbd5d8f04a
I20260808 10:43:45.824091 23517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:45.825024 23517 log.cc:824] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:45.827287 23517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15: No bootstrap required, opened a new log
I20260808 10:43:45.831480 23517 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26dd6aff9804dbaaed30dd43769ea15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 10:43:45.831764 23517 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:45.831833 23517 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26dd6aff9804dbaaed30dd43769ea15, State: Initialized, Role: FOLLOWER
I20260808 10:43:45.832342 23517 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26dd6aff9804dbaaed30dd43769ea15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 10:43:45.832535 23517 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:45.832628 23517 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:45.832751 23517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 10:43:45 dist-test-slave-ss4n krb5kdc[23434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:43:45.833782 23517 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26dd6aff9804dbaaed30dd43769ea15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 10:43:45.834137 23517 leader_election.cc:302] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26dd6aff9804dbaaed30dd43769ea15; no voters:
I20260808 10:43:45.834520 23517 leader_election.cc:288] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:45.834582 23522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:45.834775 23522 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [term 1 LEADER]: Becoming Leader. State: Replica: a26dd6aff9804dbaaed30dd43769ea15, State: Running, Role: LEADER
I20260808 10:43:45.835069 23522 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26dd6aff9804dbaaed30dd43769ea15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 10:43:45.835788 23517 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:45.836207 23523 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26dd6aff9804dbaaed30dd43769ea15. Latest consensus state: current_term: 1 leader_uuid: "a26dd6aff9804dbaaed30dd43769ea15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26dd6aff9804dbaaed30dd43769ea15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260808 10:43:45.836211 23522 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26dd6aff9804dbaaed30dd43769ea15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26dd6aff9804dbaaed30dd43769ea15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260808 10:43:45.836354 23523 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:45.836357 23522 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:45.839234 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:45.824840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45111 (local address 127.0.0.1:38267)
0808 10:43:45.825263 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:45.825278 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:45.825314 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:43:45.826211 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:45.826231 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:45.826572 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:45.826847 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:45.827762 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:45.828724 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:45.830041 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:45.830526 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:45.830654 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:45.833727 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:45.833732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:45.833745 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:45.833771 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:45.835710 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:45.836591 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:45.836599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:45.836605 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:45.836759 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:45.837142 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:45.837146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:45.837148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:45.837695 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:45.837932 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:45.838258 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 10:43:45.838498 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":144,"threads_started":1}
W20260808 10:43:45.841091 23537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:43:45.841185 23537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:43:45.841300 23526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:45.842635 23526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:45.842631 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:45.848965 23526 catalog_manager.cc:1347] Generated new cluster ID: ba4cb9498320444da5e080d79e1e99d0
I20260808 10:43:45.849041 23526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:45.856629 23526 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:45.857045 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:45.846905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57429 (local address 127.0.0.1:38267)
0808 10:43:45.847048 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:45.847053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:45.847445 (+ 392us) server_negotiation.cc:407] Connection header received
0808 10:43:45.847615 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:45.847618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:45.847667 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:45.847719 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:45.848517 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:45.849293 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:45.850546 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:45.850889 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:45.850985 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:45.852963 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:45.852967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:45.852970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:45.852995 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:45.854844 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:45.855446 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:45.855449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:45.855451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:45.855546 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:45.855961 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:45.855967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:45.855971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:45.856194 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:45.856340 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:45.856791 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 10:43:45.856894 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:43:45.857861 23526 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:45.868834 23526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26dd6aff9804dbaaed30dd43769ea15: Generated new TSK 0
I20260808 10:43:45.869649 23526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:45.875883 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23447
W20260808 10:43:45.883020 23432 connection.cc:535] client connection to 127.0.0.1:38267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38267): Connection reset by peer (error 104)
2026-08-08T10:43:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:45 dist-test-slave-ss4n krb5kdc[23548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:45 dist-test-slave-ss4n krb5kdc[23548](info): set up 2 sockets
Aug 08 10:43:45 dist-test-slave-ss4n krb5kdc[23548](info): commencing operation
krb5kdc: starting...
W20260808 10:43:46.994383 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:47 dist-test-slave-ss4n krb5kdc[23548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:47.125244 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:47.238904 23561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:47.239171 23561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:47.239212 23561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:47.243259 23561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:47.243321 23561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:47.243341 23561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:47.243360 23561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:47.243376 23561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:47.248821 23561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:47.249861 23561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:47.250803 23561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:47.259369 23567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:47.259358 23566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:47.259411 23569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:47.259753 23561 server_base.cc:1034] running on GCE node
I20260808 10:43:47.260365 23561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:47.261610 23561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:47.262838 23561 hybrid_clock.cc:648] HybridClock initialized: now 1786185827262819 us; error 39 us; skew 500 ppm
Aug 08 10:43:47 dist-test-slave-ss4n krb5kdc[23548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:47.265825 23561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:47.266819 23561 webserver.cc:460] Webserver started at http://127.0.0.1:40489/ using document root <none> and password file <none>
I20260808 10:43:47.267426 23561 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:47.267529 23561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:47.267765 23561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:47.270071 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "d2f9ae4eb499428d8b05932e829f23cd"
format_stamp: "Formatted at 2026-08-08 10:43:47 on dist-test-slave-ss4n"
server_key: "24a9909b32e757046865dd586b1aa0a3"
server_key_iv: "945c8b813608046ab5ea9f848328e6b9"
server_key_version: "encryptionkey@0"
I20260808 10:43:47.270709 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "d2f9ae4eb499428d8b05932e829f23cd"
format_stamp: "Formatted at 2026-08-08 10:43:47 on dist-test-slave-ss4n"
server_key: "24a9909b32e757046865dd586b1aa0a3"
server_key_iv: "945c8b813608046ab5ea9f848328e6b9"
server_key_version: "encryptionkey@0"
I20260808 10:43:47.274688 23561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 10:43:47.277284 23576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:47.278658 23561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:43:47.278820 23561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "d2f9ae4eb499428d8b05932e829f23cd"
format_stamp: "Formatted at 2026-08-08 10:43:47 on dist-test-slave-ss4n"
server_key: "24a9909b32e757046865dd586b1aa0a3"
server_key_iv: "945c8b813608046ab5ea9f848328e6b9"
server_key_version: "encryptionkey@0"
I20260808 10:43:47.278946 23561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:47.299719 23561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:47.303346 23561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:47.303587 23561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:47.311712 23561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36611
I20260808 10:43:47.311741 23628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36611 every 8 connection(s)
I20260808 10:43:47.312795 23561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:47.315747 23629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:47.321830 23629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd: Bootstrap starting.
I20260808 10:43:47.322515 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23561
I20260808 10:43:47.322634 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:47.322942 1801 external_mini_cluster.cc:1428] Setting key 0e83bab118cd7d2e424ff77241308a89
I20260808 10:43:47.324988 23629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:47.325919 23629 log.cc:824] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:47.328183 23629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd: No bootstrap required, opened a new log
I20260808 10:43:47.332178 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:47.324393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32895 (local address 127.0.0.1:36611)
0808 10:43:47.324880 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:47.324895 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:47.324930 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:43:47.325693 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:47.325713 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:47.326045 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:47.326431 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:47.326888 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:47.327858 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:47.328842 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:47.329312 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:47.329495 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:47.330462 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:47.330467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:47.330480 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:47.330870 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:47.330921 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:47.330943 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:47.331089 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:47.331283 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 10:43:47.331423 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":155,"threads_started":1}
I20260808 10:43:47.332453 23629 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f9ae4eb499428d8b05932e829f23cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36611 } }
I20260808 10:43:47.332722 23629 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:47.332818 23629 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f9ae4eb499428d8b05932e829f23cd, State: Initialized, Role: FOLLOWER
I20260808 10:43:47.333379 23629 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f9ae4eb499428d8b05932e829f23cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36611 } }
I20260808 10:43:47.333593 23629 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:47.333681 23629 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:47.333791 23629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:47.335224 23629 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f9ae4eb499428d8b05932e829f23cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36611 } }
I20260808 10:43:47.335631 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:47.335618 23629 leader_election.cc:302] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2f9ae4eb499428d8b05932e829f23cd; no voters:
I20260808 10:43:47.335995 23629 leader_election.cc:288] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:47.336103 23634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:47.336382 23634 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [term 1 LEADER]: Becoming Leader. State: Replica: d2f9ae4eb499428d8b05932e829f23cd, State: Running, Role: LEADER
I20260808 10:43:47.336763 23634 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f9ae4eb499428d8b05932e829f23cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36611 } }
I20260808 10:43:47.337472 23629 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:47.338235 23636 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f9ae4eb499428d8b05932e829f23cd. Latest consensus state: current_term: 1 leader_uuid: "d2f9ae4eb499428d8b05932e829f23cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f9ae4eb499428d8b05932e829f23cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36611 } } }
I20260808 10:43:47.338487 23636 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:47.338446 23635 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2f9ae4eb499428d8b05932e829f23cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f9ae4eb499428d8b05932e829f23cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36611 } } }
I20260808 10:43:47.338645 23635 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:47.339087 23647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:47.342674 23647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:47.343092 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:47.338049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45623 (local address 127.0.0.1:36611)
0808 10:43:47.338168 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:47.338173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:47.338399 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:43:47.338588 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:47.338594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:47.338644 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:47.338698 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:47.339136 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:47.339859 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:47.341086 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:47.341433 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:47.341557 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:47.342540 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:47.342544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:47.342546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:47.342559 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:47.342613 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:47.342625 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:47.342738 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:47.342906 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 10:43:47.342949 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:43:47.349320 23647 catalog_manager.cc:1347] Generated new cluster ID: bd71921642204cc086e19ac3aca5e713
I20260808 10:43:47.349416 23647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:47.355168 23647 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:47.356024 23647 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:47.367532 23647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f9ae4eb499428d8b05932e829f23cd: Generated new TSK 0
I20260808 10:43:47.368295 23647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:47.375710 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23561
W20260808 10:43:47.382234 23546 connection.cc:535] client connection to 127.0.0.1:36611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36611): Connection reset by peer (error 104)
2026-08-08T10:43:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:47 dist-test-slave-ss4n krb5kdc[23660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:47 dist-test-slave-ss4n krb5kdc[23660](info): set up 2 sockets
Aug 08 10:43:47 dist-test-slave-ss4n krb5kdc[23660](info): commencing operation
krb5kdc: starting...
W20260808 10:43:48.459904 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:48 dist-test-slave-ss4n krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:48.593613 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:48.704172 23674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:48.704432 23674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:48.704473 23674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:48.708518 23674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:48.708582 23674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:48.708602 23674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:48.708624 23674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:48.708640 23674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:48.713932 23674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:48.714968 23674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:48.716002 23674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:48.724424 23679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:48.724424 23682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:48.724500 23680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:48.724663 23674 server_base.cc:1034] running on GCE node
I20260808 10:43:48.725250 23674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:48.726425 23674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:48.727653 23674 hybrid_clock.cc:648] HybridClock initialized: now 1786185828727629 us; error 47 us; skew 500 ppm
Aug 08 10:43:48 dist-test-slave-ss4n krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:48.731966 23674 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:48.732939 23674 webserver.cc:460] Webserver started at http://127.0.0.1:44661/ using document root <none> and password file <none>
I20260808 10:43:48.733467 23674 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:48.733603 23674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:48.733790 23674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:48.735778 23674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "f40f3756e54f45f0a5106c50e66211b2"
format_stamp: "Formatted at 2026-08-08 10:43:48 on dist-test-slave-ss4n"
server_key: "05462cb974e021d79383d745d7ba7690"
server_key_iv: "7afba67a707e7c09ce59600fe29ea11d"
server_key_version: "encryptionkey@0"
I20260808 10:43:48.736282 23674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "f40f3756e54f45f0a5106c50e66211b2"
format_stamp: "Formatted at 2026-08-08 10:43:48 on dist-test-slave-ss4n"
server_key: "05462cb974e021d79383d745d7ba7690"
server_key_iv: "7afba67a707e7c09ce59600fe29ea11d"
server_key_version: "encryptionkey@0"
I20260808 10:43:48.740480 23674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 10:43:48.742986 23689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:48.744376 23674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:43:48.744602 23674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "f40f3756e54f45f0a5106c50e66211b2"
format_stamp: "Formatted at 2026-08-08 10:43:48 on dist-test-slave-ss4n"
server_key: "05462cb974e021d79383d745d7ba7690"
server_key_iv: "7afba67a707e7c09ce59600fe29ea11d"
server_key_version: "encryptionkey@0"
I20260808 10:43:48.744752 23674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:48.750084 23674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:48.753619 23674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:48.753834 23674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:48.761711 23674 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39823
I20260808 10:43:48.761761 23746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39823 every 8 connection(s)
I20260808 10:43:48.762784 23674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:48.765714 23747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:48.770846 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23674
I20260808 10:43:48.771003 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:48.771306 1801 external_mini_cluster.cc:1428] Setting key 2f6c06935eca0bfdb9a9fd6ffd905cba
I20260808 10:43:48.771287 23747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2: Bootstrap starting.
I20260808 10:43:48.773815 23747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:48.774551 23747 log.cc:824] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:48.776651 23747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2: No bootstrap required, opened a new log
I20260808 10:43:48.777060 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:48.772799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51989 (local address 127.0.0.1:39823)
0808 10:43:48.773267 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:48.773283 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:48.773320 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:43:48.774141 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:48.774162 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:48.774456 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:48.774831 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:48.775734 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:48.775739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:48.775751 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:48.775762 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:48.775818 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:48.775839 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:48.775967 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:48.776115 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 10:43:48.776282 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":165,"threads_started":1}
I20260808 10:43:48.780190 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:48.780906 23747 raft_consensus.cc:348] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40f3756e54f45f0a5106c50e66211b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39823 } }
I20260808 10:43:48.781121 23747 raft_consensus.cc:374] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:48.781153 23747 raft_consensus.cc:729] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40f3756e54f45f0a5106c50e66211b2, State: Initialized, Role: FOLLOWER
I20260808 10:43:48.781661 23747 consensus_queue.cc:260] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40f3756e54f45f0a5106c50e66211b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39823 } }
I20260808 10:43:48.781787 23747 raft_consensus.cc:388] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:48.781916 23747 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:48.782011 23747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:48.783264 23747 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40f3756e54f45f0a5106c50e66211b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39823 } }
I20260808 10:43:48.783618 23747 leader_election.cc:302] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40f3756e54f45f0a5106c50e66211b2; no voters:
I20260808 10:43:48.784005 23747 leader_election.cc:288] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:48.784060 23757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:48.784276 23757 raft_consensus.cc:686] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [term 1 LEADER]: Becoming Leader. State: Replica: f40f3756e54f45f0a5106c50e66211b2, State: Running, Role: LEADER
I20260808 10:43:48.784442 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:48.782662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60399 (local address 127.0.0.1:39823)
0808 10:43:48.782786 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:48.782800 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:48.782962 (+ 162us) server_negotiation.cc:407] Connection header received
0808 10:43:48.783086 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:48.783089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:48.783119 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:48.783189 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:48.783933 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:48.783936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:48.783939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:48.783950 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:48.783975 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:48.783986 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:48.784155 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:48.784191 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 10:43:48.784231 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:43:48.784605 23757 consensus_queue.cc:237] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40f3756e54f45f0a5106c50e66211b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39823 } }
I20260808 10:43:48.785255 23747 sys_catalog.cc:564] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:48.785748 23758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f40f3756e54f45f0a5106c50e66211b2. Latest consensus state: current_term: 1 leader_uuid: "f40f3756e54f45f0a5106c50e66211b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40f3756e54f45f0a5106c50e66211b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39823 } } }
I20260808 10:43:48.785758 23757 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f40f3756e54f45f0a5106c50e66211b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40f3756e54f45f0a5106c50e66211b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39823 } } }
I20260808 10:43:48.786012 23758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:48.786118 23757 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:48.786607 23767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:48.790041 23767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:48.795338 23767 catalog_manager.cc:1347] Generated new cluster ID: a94988eee33246f884ab4602e411ddef
I20260808 10:43:48.795413 23767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:48.806391 23767 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:48.806787 23767 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:48.814373 23767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f40f3756e54f45f0a5106c50e66211b2: Generated new TSK 0
I20260808 10:43:48.815109 23767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:48.822541 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23674
2026-08-08T10:43:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:48 dist-test-slave-ss4n krb5kdc[23780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:48 dist-test-slave-ss4n krb5kdc[23780](info): set up 2 sockets
Aug 08 10:43:48 dist-test-slave-ss4n krb5kdc[23780](info): commencing operation
krb5kdc: starting...
W20260808 10:43:49.907296 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:50 dist-test-slave-ss4n krb5kdc[23780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:50.050863 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:50.170945 23793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:50.171311 23793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:50.171406 23793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:50.175992 23793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:50.176092 23793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:50.176139 23793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:50.176211 23793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:50.176257 23793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:50.182219 23793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:50.183302 23793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:50.184340 23793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:43:50.193033 23793 server_base.cc:1034] running on GCE node
W20260808 10:43:50.192987 23801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:50.192981 23799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:50.192979 23798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:50.193846 23793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:50.194974 23793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:50.196184 23793 hybrid_clock.cc:648] HybridClock initialized: now 1786185830196155 us; error 43 us; skew 500 ppm
Aug 08 10:43:50 dist-test-slave-ss4n krb5kdc[23780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:50.204981 23793 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:50.206012 23793 webserver.cc:460] Webserver started at http://127.0.0.1:35767/ using document root <none> and password file <none>
I20260808 10:43:50.206602 23793 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:50.206698 23793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:50.206938 23793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:50.209133 23793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "18efa09c51bb440e94c8035674ebe499"
format_stamp: "Formatted at 2026-08-08 10:43:50 on dist-test-slave-ss4n"
server_key: "fa296218cfc1d98294d0352739ee7233"
server_key_iv: "ab63a091f9576e33cd57d3f0f22441c6"
server_key_version: "encryptionkey@0"
I20260808 10:43:50.209772 23793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "18efa09c51bb440e94c8035674ebe499"
format_stamp: "Formatted at 2026-08-08 10:43:50 on dist-test-slave-ss4n"
server_key: "fa296218cfc1d98294d0352739ee7233"
server_key_iv: "ab63a091f9576e33cd57d3f0f22441c6"
server_key_version: "encryptionkey@0"
I20260808 10:43:50.213394 23793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:50.215826 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:50.216945 23793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:50.217116 23793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "18efa09c51bb440e94c8035674ebe499"
format_stamp: "Formatted at 2026-08-08 10:43:50 on dist-test-slave-ss4n"
server_key: "fa296218cfc1d98294d0352739ee7233"
server_key_iv: "ab63a091f9576e33cd57d3f0f22441c6"
server_key_version: "encryptionkey@0"
I20260808 10:43:50.217239 23793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:50.228446 23793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:50.231833 23793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:50.232055 23793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:50.239418 23793 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39935
I20260808 10:43:50.239441 23863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39935 every 8 connection(s)
I20260808 10:43:50.240386 23793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:50.243119 23864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:50.248615 23864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499: Bootstrap starting.
I20260808 10:43:50.248996 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23793
I20260808 10:43:50.249146 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:50.249425 1801 external_mini_cluster.cc:1428] Setting key d0034832e5ebf3a8befa1f0d13c45819
I20260808 10:43:50.251325 23864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:50.252050 23864 log.cc:824] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:50.254304 23864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499: No bootstrap required, opened a new log
Aug 08 10:43:50 dist-test-slave-ss4n krb5kdc[23780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:43:50.259011 23864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18efa09c51bb440e94c8035674ebe499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260808 10:43:50.259312 23864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:50.259397 23864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18efa09c51bb440e94c8035674ebe499, State: Initialized, Role: FOLLOWER
I20260808 10:43:50.259889 23864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18efa09c51bb440e94c8035674ebe499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260808 10:43:50.260092 23864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:50.260193 23864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:50.260298 23864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:50.261564 23864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18efa09c51bb440e94c8035674ebe499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260808 10:43:50.261919 23864 leader_election.cc:302] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18efa09c51bb440e94c8035674ebe499; no voters:
I20260808 10:43:50.262291 23864 leader_election.cc:288] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:50.262396 23869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:50.262612 23869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [term 1 LEADER]: Becoming Leader. State: Replica: 18efa09c51bb440e94c8035674ebe499, State: Running, Role: LEADER
I20260808 10:43:50.263042 23869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18efa09c51bb440e94c8035674ebe499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } }
I20260808 10:43:50.263703 23864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:50.264678 23870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18efa09c51bb440e94c8035674ebe499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18efa09c51bb440e94c8035674ebe499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } } }
I20260808 10:43:50.264868 23870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:50.265285 23871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18efa09c51bb440e94c8035674ebe499. Latest consensus state: current_term: 1 leader_uuid: "18efa09c51bb440e94c8035674ebe499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18efa09c51bb440e94c8035674ebe499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39935 } } }
I20260808 10:43:50.265430 23871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:50.266235 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:50.250810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58299 (local address 127.0.0.1:39935)
0808 10:43:50.251244 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:50.251258 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:50.251292 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:43:50.251978 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:50.251995 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:50.252416 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:50.252791 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:50.253805 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:50.254847 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:50.256066 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:50.256625 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:50.256744 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:50.259707 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:50.259713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:50.259728 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:50.259759 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:50.262131 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:50.262755 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:50.262761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:50.262767 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:50.262912 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:50.263271 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:50.263274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:50.263276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:50.263772 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:50.264606 (+ 834us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:50.264793 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 10:43:50.265108 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":132,"threads_started":1}
W20260808 10:43:50.268832 23884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:43:50.268923 23884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:43:50.269071 23878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:50.269538 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:50.269984 23878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:50.275535 23878 catalog_manager.cc:1347] Generated new cluster ID: aec53abb73e746fab7f0daf131f4aec2
I20260808 10:43:50.275945 23878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:50.280061 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:50.271515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34129 (local address 127.0.0.1:39935)
0808 10:43:50.271610 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:50.271616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:50.271968 (+ 352us) server_negotiation.cc:407] Connection header received
0808 10:43:50.272113 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:50.272116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:50.272158 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:50.272253 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:50.272987 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:50.273635 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:50.274730 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:50.275048 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:50.275123 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:50.276755 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:50.276759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:50.276761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:50.276785 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:50.278330 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:50.278773 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:50.278777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:50.278779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:50.278863 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:50.279168 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:50.279172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:50.279173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:50.279354 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:50.279515 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:50.279755 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 10:43:50.279869 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:43:50.307343 23878 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:50.308380 23878 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:50.318373 23878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18efa09c51bb440e94c8035674ebe499: Generated new TSK 0
I20260808 10:43:50.319134 23878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:50.325237 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23793
2026-08-08T10:43:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:50 dist-test-slave-ss4n krb5kdc[23895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:50 dist-test-slave-ss4n krb5kdc[23895](info): set up 2 sockets
Aug 08 10:43:50 dist-test-slave-ss4n krb5kdc[23895](info): commencing operation
krb5kdc: starting...
W20260808 10:43:51.439720 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:51 dist-test-slave-ss4n krb5kdc[23895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:51.572815 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:51.685497 23908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:51.685840 23908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:51.685926 23908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:51.690024 23908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:51.690132 23908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:51.690172 23908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:51.690229 23908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:51.690268 23908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:51.695611 23908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:51.696746 23908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:51.697824 23908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:51.706318 23917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:51.706457 23908 server_base.cc:1034] running on GCE node
W20260808 10:43:51.706638 23915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:51.706286 23913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:51.707329 23908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:51.708678 23908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:51.709924 23908 hybrid_clock.cc:648] HybridClock initialized: now 1786185831709893 us; error 45 us; skew 500 ppm
Aug 08 10:43:51 dist-test-slave-ss4n krb5kdc[23895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:51.712872 23908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:51.713833 23908 webserver.cc:460] Webserver started at http://127.0.0.1:33909/ using document root <none> and password file <none>
I20260808 10:43:51.714366 23908 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:51.714427 23908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:51.714599 23908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:51.716570 23908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "3724e3dc34d14de0af4d0f8cc2166eee"
format_stamp: "Formatted at 2026-08-08 10:43:51 on dist-test-slave-ss4n"
server_key: "c0fa2a87125609a983b11b93aab593f9"
server_key_iv: "d404b26c0c1d43eeb5f2bcef8d5e9446"
server_key_version: "encryptionkey@0"
I20260808 10:43:51.717103 23908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "3724e3dc34d14de0af4d0f8cc2166eee"
format_stamp: "Formatted at 2026-08-08 10:43:51 on dist-test-slave-ss4n"
server_key: "c0fa2a87125609a983b11b93aab593f9"
server_key_iv: "d404b26c0c1d43eeb5f2bcef8d5e9446"
server_key_version: "encryptionkey@0"
I20260808 10:43:51.720798 23908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:43:51.723250 23924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:51.724534 23908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:43:51.724769 23908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "3724e3dc34d14de0af4d0f8cc2166eee"
format_stamp: "Formatted at 2026-08-08 10:43:51 on dist-test-slave-ss4n"
server_key: "c0fa2a87125609a983b11b93aab593f9"
server_key_iv: "d404b26c0c1d43eeb5f2bcef8d5e9446"
server_key_version: "encryptionkey@0"
I20260808 10:43:51.724881 23908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:51.744074 23908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:51.747352 23908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:51.747548 23908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:51.754643 23908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42131
I20260808 10:43:51.754678 23979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42131 every 8 connection(s)
I20260808 10:43:51.755591 23908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:51.758363 23980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:51.759752 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 23908
I20260808 10:43:51.759895 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:51.760207 1801 external_mini_cluster.cc:1428] Setting key ead000ad387c2383a99b31b9809fb9d3
I20260808 10:43:51.765257 23980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee: Bootstrap starting.
I20260808 10:43:51.767838 23980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:51.768721 23980 log.cc:824] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:51.770265 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:51.761640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43223 (local address 127.0.0.1:42131)
0808 10:43:51.762065 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:51.762085 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:51.762126 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:43:51.762955 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:51.762978 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:51.763456 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:51.763810 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:51.764232 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:51.765666 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:51.766566 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:51.767227 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:51.767382 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:51.768265 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:51.768271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:51.768285 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:51.768783 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:51.768851 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:51.768880 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:51.768975 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:51.769089 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 10:43:51.769297 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260808 10:43:51.770980 23980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee: No bootstrap required, opened a new log
I20260808 10:43:51.773042 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:51.774845 23980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260808 10:43:51.775117 23980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:51.775261 23980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3724e3dc34d14de0af4d0f8cc2166eee, State: Initialized, Role: FOLLOWER
I20260808 10:43:51.775770 23980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260808 10:43:51.775964 23980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:51.776032 23980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:51.776126 23980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:51.777169 23980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260808 10:43:51.777554 23980 leader_election.cc:302] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3724e3dc34d14de0af4d0f8cc2166eee; no voters:
I20260808 10:43:51.777885 23980 leader_election.cc:288] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:51.777987 23990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:51.778251 23990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [term 1 LEADER]: Becoming Leader. State: Replica: 3724e3dc34d14de0af4d0f8cc2166eee, State: Running, Role: LEADER
I20260808 10:43:51.778582 23990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260808 10:43:51.779345 23980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:51.779515 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:51.775101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37079 (local address 127.0.0.1:42131)
0808 10:43:51.775227 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:51.775233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:51.775416 (+ 183us) server_negotiation.cc:407] Connection header received
0808 10:43:51.775535 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:51.775538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:51.775584 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:51.775666 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:51.775904 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:51.776661 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:51.777566 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:51.777961 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:51.778066 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:51.779048 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:51.779051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:51.779055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:51.779067 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:51.779094 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:51.779106 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:51.779152 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:51.779323 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 10:43:51.779383 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:43:51.780206 23991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3724e3dc34d14de0af4d0f8cc2166eee. Latest consensus state: current_term: 1 leader_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } } }
I20260808 10:43:51.780339 23991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:51.780481 23992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3724e3dc34d14de0af4d0f8cc2166eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } } }
I20260808 10:43:51.780561 23992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee [sys.catalog]: This master's current role is: LEADER
W20260808 10:43:51.784090 24005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:43:51.784170 24005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:43:51.784231 23999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:51.784848 23999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:51.790122 23999 catalog_manager.cc:1347] Generated new cluster ID: b1ce349d2b1a4dd0ab505f31b19cf745
I20260808 10:43:51.790187 23999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:51.801793 23999 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:51.802701 23999 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:51.825155 23999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3724e3dc34d14de0af4d0f8cc2166eee: Generated new TSK 0
I20260808 10:43:51.825989 23999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:51.833103 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 23908
2026-08-08T10:43:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:51 dist-test-slave-ss4n krb5kdc[24011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:51 dist-test-slave-ss4n krb5kdc[24011](info): set up 2 sockets
Aug 08 10:43:51 dist-test-slave-ss4n krb5kdc[24011](info): commencing operation
krb5kdc: starting...
W20260808 10:43:52.911708 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:53 dist-test-slave-ss4n krb5kdc[24011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:53.047046 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46577
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:53.158592 24024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:53.158856 24024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:53.158898 24024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:53.163120 24024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:53.163182 24024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:53.163201 24024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:53.163218 24024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:53.163264 24024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:53.168529 24024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:53.169601 24024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:53.170595 24024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:53.179198 24030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:53.179205 24032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:53.179283 24029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:53.179410 24024 server_base.cc:1034] running on GCE node
I20260808 10:43:53.180099 24024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:53.181216 24024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:53.182415 24024 hybrid_clock.cc:648] HybridClock initialized: now 1786185833182369 us; error 68 us; skew 500 ppm
Aug 08 10:43:53 dist-test-slave-ss4n krb5kdc[24011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:53.185197 24024 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:53.186139 24024 webserver.cc:460] Webserver started at http://127.0.0.1:42601/ using document root <none> and password file <none>
I20260808 10:43:53.186712 24024 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:53.186808 24024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:53.187038 24024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:53.189041 24024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "5e9fe3ac0b144b10867bc214c3c65324"
format_stamp: "Formatted at 2026-08-08 10:43:53 on dist-test-slave-ss4n"
server_key: "c52a05c82101cf9da0fcddc6867fbf4d"
server_key_iv: "633ba5d762e600ed7f176ce85db4140c"
server_key_version: "encryptionkey@0"
I20260808 10:43:53.189672 24024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "5e9fe3ac0b144b10867bc214c3c65324"
format_stamp: "Formatted at 2026-08-08 10:43:53 on dist-test-slave-ss4n"
server_key: "c52a05c82101cf9da0fcddc6867fbf4d"
server_key_iv: "633ba5d762e600ed7f176ce85db4140c"
server_key_version: "encryptionkey@0"
I20260808 10:43:53.193231 24024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:43:53.195691 24039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:53.196899 24024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:43:53.197048 24024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "5e9fe3ac0b144b10867bc214c3c65324"
format_stamp: "Formatted at 2026-08-08 10:43:53 on dist-test-slave-ss4n"
server_key: "c52a05c82101cf9da0fcddc6867fbf4d"
server_key_iv: "633ba5d762e600ed7f176ce85db4140c"
server_key_version: "encryptionkey@0"
I20260808 10:43:53.197170 24024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:53.201733 24024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:53.204634 24024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:53.204831 24024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:53.212011 24024 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46577
I20260808 10:43:53.212037 24093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46577 every 8 connection(s)
I20260808 10:43:53.212945 24024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:53.214315 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24024
I20260808 10:43:53.214465 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:53.214811 1801 external_mini_cluster.cc:1428] Setting key ef002fe20b2be5b78ad6f7ecac559567
I20260808 10:43:53.216043 24094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:53.220427 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:53.216278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33317 (local address 127.0.0.1:46577)
0808 10:43:53.216681 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:53.216700 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:53.216740 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:43:53.217612 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:53.217634 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:53.217975 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:53.218329 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:53.219058 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:53.219064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:53.219078 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:53.219089 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:53.219153 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:53.219178 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:53.219319 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:53.219386 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 10:43:53.219529 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":99,"threads_started":1}
I20260808 10:43:53.223022 24094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324: Bootstrap starting.
I20260808 10:43:53.223289 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:53.225306 24094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:53.226083 24094 log.cc:824] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:53.227165 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:53.225402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60753 (local address 127.0.0.1:46577)
0808 10:43:53.225531 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:53.225535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:53.225751 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:43:53.225890 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:53.225893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:53.225931 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:53.226012 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:53.226767 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:53.226771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:53.226773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:53.226784 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:53.226810 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:53.226822 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:53.226916 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:53.226999 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 10:43:53.227035 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:43:53.228142 24094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324: No bootstrap required, opened a new log
I20260808 10:43:53.231594 24094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9fe3ac0b144b10867bc214c3c65324" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46577 } }
I20260808 10:43:53.231874 24094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:53.231948 24094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e9fe3ac0b144b10867bc214c3c65324, State: Initialized, Role: FOLLOWER
I20260808 10:43:53.232409 24094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9fe3ac0b144b10867bc214c3c65324" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46577 } }
I20260808 10:43:53.232555 24094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:53.232630 24094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:53.232726 24094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:53.233702 24094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9fe3ac0b144b10867bc214c3c65324" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46577 } }
I20260808 10:43:53.234033 24094 leader_election.cc:302] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e9fe3ac0b144b10867bc214c3c65324; no voters:
I20260808 10:43:53.234338 24094 leader_election.cc:288] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:53.234472 24104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:53.234715 24104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [term 1 LEADER]: Becoming Leader. State: Replica: 5e9fe3ac0b144b10867bc214c3c65324, State: Running, Role: LEADER
I20260808 10:43:53.235091 24104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9fe3ac0b144b10867bc214c3c65324" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46577 } }
I20260808 10:43:53.235592 24094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:53.236716 24106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e9fe3ac0b144b10867bc214c3c65324. Latest consensus state: current_term: 1 leader_uuid: "5e9fe3ac0b144b10867bc214c3c65324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9fe3ac0b144b10867bc214c3c65324" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46577 } } }
I20260808 10:43:53.236855 24106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:53.237151 24105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e9fe3ac0b144b10867bc214c3c65324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9fe3ac0b144b10867bc214c3c65324" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46577 } } }
I20260808 10:43:53.237239 24114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:53.237340 24105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:53.240569 24114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:53.246465 24114 catalog_manager.cc:1347] Generated new cluster ID: 49801446d3684730bdcb919febbafd40
I20260808 10:43:53.246577 24114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:53.266155 24114 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:53.266492 24114 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:53.283274 24114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e9fe3ac0b144b10867bc214c3c65324: Generated new TSK 0
I20260808 10:43:53.284114 24114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:53.303109 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24024
2026-08-08T10:43:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:53 dist-test-slave-ss4n krb5kdc[24127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:53 dist-test-slave-ss4n krb5kdc[24127](info): set up 2 sockets
Aug 08 10:43:53 dist-test-slave-ss4n krb5kdc[24127](info): commencing operation
krb5kdc: starting...
W20260808 10:43:54.392547 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:54 dist-test-slave-ss4n krb5kdc[24127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:54.529098 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:54.641265 24140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:54.641614 24140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:54.641701 24140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:54.645733 24140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:54.645836 24140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:54.645875 24140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:54.645915 24140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:54.645949 24140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:54.651216 24140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:54.652285 24140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:54.653281 24140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:54.661939 24146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:54.662117 24140 server_base.cc:1034] running on GCE node
W20260808 10:43:54.662002 24145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:54.661942 24148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:54.662779 24140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:54.663882 24140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:54.665130 24140 hybrid_clock.cc:648] HybridClock initialized: now 1786185834665104 us; error 37 us; skew 500 ppm
Aug 08 10:43:54 dist-test-slave-ss4n krb5kdc[24127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:54.668140 24140 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:54.669044 24140 webserver.cc:460] Webserver started at http://127.0.0.1:37483/ using document root <none> and password file <none>
I20260808 10:43:54.669715 24140 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:54.669790 24140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:54.670025 24140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:54.672091 24140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "56586a79612643e6a5d0f0bec9d4f1e3"
format_stamp: "Formatted at 2026-08-08 10:43:54 on dist-test-slave-ss4n"
server_key: "20f5a7fb84834ff29ed1f312cad64466"
server_key_iv: "abe037128a80b1f17cde635cf70a8663"
server_key_version: "encryptionkey@0"
I20260808 10:43:54.672662 24140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "56586a79612643e6a5d0f0bec9d4f1e3"
format_stamp: "Formatted at 2026-08-08 10:43:54 on dist-test-slave-ss4n"
server_key: "20f5a7fb84834ff29ed1f312cad64466"
server_key_iv: "abe037128a80b1f17cde635cf70a8663"
server_key_version: "encryptionkey@0"
I20260808 10:43:54.676484 24140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 10:43:54.679083 24156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:54.680337 24140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:43:54.680495 24140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "56586a79612643e6a5d0f0bec9d4f1e3"
format_stamp: "Formatted at 2026-08-08 10:43:54 on dist-test-slave-ss4n"
server_key: "20f5a7fb84834ff29ed1f312cad64466"
server_key_iv: "abe037128a80b1f17cde635cf70a8663"
server_key_version: "encryptionkey@0"
I20260808 10:43:54.680626 24140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:54.699141 24140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:54.702450 24140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:54.702689 24140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:54.710098 24140 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38275
I20260808 10:43:54.710126 24213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38275 every 8 connection(s)
I20260808 10:43:54.711143 24140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:54.713909 24214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:54.716341 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24140
I20260808 10:43:54.716455 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:54.716795 1801 external_mini_cluster.cc:1428] Setting key 0adf8dd1aea965d8b4fbd938e0fc6e4c
I20260808 10:43:54.719756 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3: Bootstrap starting.
I20260808 10:43:54.722364 24214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:54.723250 24214 log.cc:824] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:54.725320 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3: No bootstrap required, opened a new log
Aug 08 10:43:54 dist-test-slave-ss4n krb5kdc[24127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:43:54.729422 24214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38275 } }
I20260808 10:43:54.729743 24214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:54.729815 24214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56586a79612643e6a5d0f0bec9d4f1e3, State: Initialized, Role: FOLLOWER
I20260808 10:43:54.730311 24214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38275 } }
I20260808 10:43:54.730482 24214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:54.730566 24214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:54.730675 24214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:54.731887 24214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38275 } }
I20260808 10:43:54.732235 24214 leader_election.cc:302] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56586a79612643e6a5d0f0bec9d4f1e3; no voters:
I20260808 10:43:54.732577 24214 leader_election.cc:288] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:54.732685 24219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:54.732771 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:54.718365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:55675 (local address 127.0.0.1:38275)
0808 10:43:54.718808 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:54.718822 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:54.718856 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:43:54.719441 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:54.719456 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:54.719787 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:54.720113 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:54.721027 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:54.722154 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:54.723392 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:54.723870 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:54.724043 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:54.726696 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:54.726702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:54.726718 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:54.726749 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:54.729344 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:54.729970 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:54.729976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:54.729981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:54.730127 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:54.730478 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:54.730482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:54.730483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:54.731012 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:54.731235 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:54.731549 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 10:43:54.731790 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":154,"threads_started":1}
I20260808 10:43:54.732926 24219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 56586a79612643e6a5d0f0bec9d4f1e3, State: Running, Role: LEADER
I20260808 10:43:54.733424 24219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38275 } }
I20260808 10:43:54.733855 24214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:54.735033 24221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56586a79612643e6a5d0f0bec9d4f1e3. Latest consensus state: current_term: 1 leader_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38275 } } }
I20260808 10:43:54.735172 24221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:54.735489 24220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56586a79612643e6a5d0f0bec9d4f1e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38275 } } }
I20260808 10:43:54.735567 24220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:54.736411 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:54.735927 24228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:54.739706 24228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:54.745429 24228 catalog_manager.cc:1347] Generated new cluster ID: f775c822c1634613b08f33b7fc3be72d
I20260808 10:43:54.745816 24228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:54.748167 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:54.738679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:37415 (local address 127.0.0.1:38275)
0808 10:43:54.738781 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:54.738785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:54.739012 (+ 227us) server_negotiation.cc:407] Connection header received
0808 10:43:54.739112 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:54.739115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:54.739162 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:54.739252 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:54.740072 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:54.740815 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:54.742047 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:54.742435 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:54.742512 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:54.744377 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:54.744380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:54.744383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:54.744407 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:54.746183 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:54.746789 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:54.746792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:54.746794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:54.746870 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:54.747202 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:54.747205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:54.747207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:54.747385 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:54.747497 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:54.747926 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 10:43:54.748017 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:43:54.775703 24228 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:54.776993 24228 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:54.795791 24228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56586a79612643e6a5d0f0bec9d4f1e3: Generated new TSK 0
I20260808 10:43:54.796556 24228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:54.804639 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24140
W20260808 10:43:54.810945 24125 connection.cc:535] client connection to 127.0.0.1:38275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38275): Connection reset by peer (error 104)
2026-08-08T10:43:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:54 dist-test-slave-ss4n krb5kdc[24245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:54 dist-test-slave-ss4n krb5kdc[24245](info): set up 2 sockets
Aug 08 10:43:54 dist-test-slave-ss4n krb5kdc[24245](info): commencing operation
krb5kdc: starting...
W20260808 10:43:55.992058 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:56 dist-test-slave-ss4n krb5kdc[24245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:56.137315 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33393
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:56.253212 24259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:56.253504 24259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:56.253548 24259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:56.257400 24259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:56.257463 24259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:56.257505 24259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:56.257524 24259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:56.257539 24259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:56.262814 24259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:56.263805 24259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:56.264753 24259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:56.273375 24265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:56.273355 24267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:56.273643 24264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:56.273742 24259 server_base.cc:1034] running on GCE node
I20260808 10:43:56.274310 24259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:56.275461 24259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:56.276664 24259 hybrid_clock.cc:648] HybridClock initialized: now 1786185836276628 us; error 41 us; skew 500 ppm
Aug 08 10:43:56 dist-test-slave-ss4n krb5kdc[24245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:56.286983 24259 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:56.288194 24259 webserver.cc:460] Webserver started at http://127.0.0.1:36905/ using document root <none> and password file <none>
I20260808 10:43:56.288832 24259 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:56.288904 24259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:56.289147 24259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:56.291471 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "6d9222048d7e4250b9979613e85f3709"
format_stamp: "Formatted at 2026-08-08 10:43:56 on dist-test-slave-ss4n"
server_key: "4d6e47856595c01e0f13d7fd91baf3dd"
server_key_iv: "77854fe3126b285c800ed3e8271bc2a3"
server_key_version: "encryptionkey@0"
I20260808 10:43:56.292098 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "6d9222048d7e4250b9979613e85f3709"
format_stamp: "Formatted at 2026-08-08 10:43:56 on dist-test-slave-ss4n"
server_key: "4d6e47856595c01e0f13d7fd91baf3dd"
server_key_iv: "77854fe3126b285c800ed3e8271bc2a3"
server_key_version: "encryptionkey@0"
I20260808 10:43:56.296295 24259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 10:43:56.299233 24275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:56.300659 24259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 10:43:56.300838 24259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "6d9222048d7e4250b9979613e85f3709"
format_stamp: "Formatted at 2026-08-08 10:43:56 on dist-test-slave-ss4n"
server_key: "4d6e47856595c01e0f13d7fd91baf3dd"
server_key_iv: "77854fe3126b285c800ed3e8271bc2a3"
server_key_version: "encryptionkey@0"
I20260808 10:43:56.300979 24259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:56.335494 24259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:56.338873 24259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:56.339092 24259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:56.346568 24259 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33393
I20260808 10:43:56.346596 24329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33393 every 8 connection(s)
I20260808 10:43:56.347513 24259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:56.350317 24330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:56.355054 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24259
I20260808 10:43:56.355180 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:56.355469 1801 external_mini_cluster.cc:1428] Setting key 67446daf4fbfea342539fdd7bb90d9f7
I20260808 10:43:56.355727 24330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709: Bootstrap starting.
I20260808 10:43:56.358451 24330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:56.359338 24330 log.cc:824] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:56.361549 24330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709: No bootstrap required, opened a new log
I20260808 10:43:56.366024 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:56.356856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:43773 (local address 127.0.0.1:33393)
0808 10:43:56.357351 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:56.357372 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:56.357413 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:43:56.358266 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:56.358290 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:56.358742 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:56.359105 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:56.359531 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:56.360857 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:56.361973 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:56.362660 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:56.362814 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:56.363728 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:56.363734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:56.363750 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:56.364267 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:56.364347 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:56.364383 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:56.364547 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:56.364900 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:43:56.365036 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":118,"threads_started":1}
I20260808 10:43:56.366091 24330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9222048d7e4250b9979613e85f3709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33393 } }
I20260808 10:43:56.366349 24330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:56.366426 24330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9222048d7e4250b9979613e85f3709, State: Initialized, Role: FOLLOWER
I20260808 10:43:56.366937 24330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9222048d7e4250b9979613e85f3709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33393 } }
I20260808 10:43:56.367063 24330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:56.367159 24330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:56.367254 24330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:56.368461 24330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9222048d7e4250b9979613e85f3709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33393 } }
I20260808 10:43:56.368837 24330 leader_election.cc:302] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d9222048d7e4250b9979613e85f3709; no voters:
I20260808 10:43:56.369174 24330 leader_election.cc:288] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:56.369270 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:56.369283 24335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:56.369529 24335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [term 1 LEADER]: Becoming Leader. State: Replica: 6d9222048d7e4250b9979613e85f3709, State: Running, Role: LEADER
I20260808 10:43:56.369890 24335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9222048d7e4250b9979613e85f3709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33393 } }
I20260808 10:43:56.370770 24330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:56.371027 24335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d9222048d7e4250b9979613e85f3709. Latest consensus state: current_term: 1 leader_uuid: "6d9222048d7e4250b9979613e85f3709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9222048d7e4250b9979613e85f3709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33393 } } }
I20260808 10:43:56.371140 24335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:56.371470 24336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d9222048d7e4250b9979613e85f3709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9222048d7e4250b9979613e85f3709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33393 } } }
I20260808 10:43:56.371587 24336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:56.371662 24344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:56.375381 24344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:56.376937 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:56.372371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:59937 (local address 127.0.0.1:33393)
0808 10:43:56.372487 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:56.372493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:56.372513 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:43:56.372555 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:56.372560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:56.372615 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:56.372704 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:56.373023 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:56.373815 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:56.375023 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:56.375333 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:56.375450 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:56.376385 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:56.376388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:56.376391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:56.376403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:56.376430 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:56.376443 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:56.376527 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:56.376765 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 10:43:56.376798 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:43:56.380877 24344 catalog_manager.cc:1347] Generated new cluster ID: ad923ea048214530ad0a7fb95e0189c9
I20260808 10:43:56.380947 24344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:56.387020 24344 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:56.388211 24344 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:56.406805 24344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d9222048d7e4250b9979613e85f3709: Generated new TSK 0
I20260808 10:43:56.407569 24344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:56.412340 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24259
W20260808 10:43:56.419098 24243 connection.cc:535] client connection to 127.0.0.1:33393 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33393): Connection reset by peer (error 104)
2026-08-08T10:43:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1608 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:56 dist-test-slave-ss4n krb5kdc[24362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:56 dist-test-slave-ss4n krb5kdc[24362](info): set up 2 sockets
Aug 08 10:43:56 dist-test-slave-ss4n krb5kdc[24362](info): commencing operation
krb5kdc: starting...
W20260808 10:43:57.511870 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:57 dist-test-slave-ss4n krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:57.649910 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:57.761726 24375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:57.762054 24375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:57.762141 24375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:57.767115 24375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:57.767235 24375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:57.767273 24375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:57.767302 24375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:57.767326 24375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:57.773864 24375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:57.774963 24375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:57.776013 24375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:57.784672 24380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:57.784631 24381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:57.784760 24375 server_base.cc:1034] running on GCE node
W20260808 10:43:57.784745 24383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:57.785403 24375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:57.786566 24375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:57.787739 24375 hybrid_clock.cc:648] HybridClock initialized: now 1786185837787708 us; error 42 us; skew 500 ppm
Aug 08 10:43:57 dist-test-slave-ss4n krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:57.790642 24375 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:57.791477 24375 webserver.cc:460] Webserver started at http://127.0.0.1:36203/ using document root <none> and password file <none>
I20260808 10:43:57.792032 24375 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:57.792126 24375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:57.792349 24375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:57.794397 24375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "a02f9f86cfd64180b2d25dcb6a85b505"
format_stamp: "Formatted at 2026-08-08 10:43:57 on dist-test-slave-ss4n"
server_key: "f5b4b3ad216b05e2ba68642e8fa4a214"
server_key_iv: "514c364c91336005433c0726121db54d"
server_key_version: "encryptionkey@0"
I20260808 10:43:57.794981 24375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "a02f9f86cfd64180b2d25dcb6a85b505"
format_stamp: "Formatted at 2026-08-08 10:43:57 on dist-test-slave-ss4n"
server_key: "f5b4b3ad216b05e2ba68642e8fa4a214"
server_key_iv: "514c364c91336005433c0726121db54d"
server_key_version: "encryptionkey@0"
I20260808 10:43:57.798659 24375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:43:57.800890 24391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:57.802071 24375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:43:57.802222 24375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "a02f9f86cfd64180b2d25dcb6a85b505"
format_stamp: "Formatted at 2026-08-08 10:43:57 on dist-test-slave-ss4n"
server_key: "f5b4b3ad216b05e2ba68642e8fa4a214"
server_key_iv: "514c364c91336005433c0726121db54d"
server_key_version: "encryptionkey@0"
I20260808 10:43:57.802335 24375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:57.806833 24375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:57.810104 24375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:57.810303 24375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:57.817077 24375 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36853
I20260808 10:43:57.817116 24444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36853 every 8 connection(s)
I20260808 10:43:57.817992 24375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:57.820585 24445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:57.826118 24445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505: Bootstrap starting.
I20260808 10:43:57.827112 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24375
I20260808 10:43:57.827227 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:57.827557 1801 external_mini_cluster.cc:1428] Setting key df9e99870b412fc890424e04a58e883e
I20260808 10:43:57.828945 24445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:57.829761 24445 log.cc:824] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505: Log is configured to *not* fsync() on all Append() calls
I20260808 10:43:57.832003 24445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505: No bootstrap required, opened a new log
I20260808 10:43:57.832762 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:57.828752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:34587 (local address 127.0.0.1:36853)
0808 10:43:57.829230 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:57.829245 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:57.829280 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:43:57.830060 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:57.830080 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:57.830366 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:57.830641 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:57.831510 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:57.831515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:57.831526 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:57.831536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:57.831587 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:57.831607 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:57.831700 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:57.831847 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 10:43:57.832024 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":155,"threads_started":1}
I20260808 10:43:57.835731 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:57.836262 24445 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260808 10:43:57.836541 24445 raft_consensus.cc:374] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:57.836614 24445 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02f9f86cfd64180b2d25dcb6a85b505, State: Initialized, Role: FOLLOWER
I20260808 10:43:57.837077 24445 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260808 10:43:57.837263 24445 raft_consensus.cc:388] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:57.837338 24445 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:57.837440 24445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:57.838677 24445 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260808 10:43:57.839042 24445 leader_election.cc:302] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a02f9f86cfd64180b2d25dcb6a85b505; no voters:
I20260808 10:43:57.839411 24445 leader_election.cc:288] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:57.839474 24455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:57.839793 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:57.838090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56441 (local address 127.0.0.1:36853)
0808 10:43:57.838228 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:57.838232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:57.838419 (+ 187us) server_negotiation.cc:407] Connection header received
0808 10:43:57.838554 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:57.838556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:57.838584 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:57.838648 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:57.839361 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:57.839363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:57.839365 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:43:57.839375 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:57.839399 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:43:57.839408 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:57.839516 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:57.839619 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:43:57.839661 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:43:57.839803 24455 raft_consensus.cc:686] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [term 1 LEADER]: Becoming Leader. State: Replica: a02f9f86cfd64180b2d25dcb6a85b505, State: Running, Role: LEADER
I20260808 10:43:57.840255 24455 consensus_queue.cc:237] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260808 10:43:57.840749 24445 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:57.841315 24455 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02f9f86cfd64180b2d25dcb6a85b505. Latest consensus state: current_term: 1 leader_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } } }
I20260808 10:43:57.841398 24455 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:57.841802 24457 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f9f86cfd64180b2d25dcb6a85b505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } } }
I20260808 10:43:57.841920 24457 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505 [sys.catalog]: This master's current role is: LEADER
W20260808 10:43:57.845458 24470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:43:57.845615 24470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:43:57.845693 24460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:57.846707 24460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:57.852240 24460 catalog_manager.cc:1347] Generated new cluster ID: b2f2da337fc74c898b58e3e340ebf82c
I20260808 10:43:57.852319 24460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:57.864208 24460 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:57.864543 24460 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:57.872387 24460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a02f9f86cfd64180b2d25dcb6a85b505: Generated new TSK 0
I20260808 10:43:57.873147 24460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:57.879514 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24375
2026-08-08T10:43:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:57 dist-test-slave-ss4n krb5kdc[24476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:57 dist-test-slave-ss4n krb5kdc[24476](info): set up 2 sockets
Aug 08 10:43:57 dist-test-slave-ss4n krb5kdc[24476](info): commencing operation
krb5kdc: starting...
W20260808 10:43:58.971825 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:43:59 dist-test-slave-ss4n krb5kdc[24476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:43:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:43:59.109402 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:43:59.226572 24490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:43:59.226926 24490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:43:59.227020 24490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:43:59.231607 24490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:43:59.231724 24490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:43:59.231767 24490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:43:59.231806 24490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:43:59.231849 24490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:43:59.238056 24490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:43:59.239283 24490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:43:59.240397 24490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:43:59.249470 24495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:59.249467 24496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:43:59.249706 24498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:43:59.250097 24490 server_base.cc:1034] running on GCE node
I20260808 10:43:59.250640 24490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:43:59.251729 24490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:43:59.252900 24490 hybrid_clock.cc:648] HybridClock initialized: now 1786185839252869 us; error 45 us; skew 500 ppm
Aug 08 10:43:59 dist-test-slave-ss4n krb5kdc[24476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:43:59.255788 24490 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:43:59.256757 24490 webserver.cc:460] Webserver started at http://127.0.0.1:36447/ using document root <none> and password file <none>
I20260808 10:43:59.257320 24490 fs_manager.cc:359] Metadata directory not provided
I20260808 10:43:59.257416 24490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:43:59.257669 24490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:43:59.259696 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "4d645fd1548d4f04a3d52948978c149a"
format_stamp: "Formatted at 2026-08-08 10:43:59 on dist-test-slave-ss4n"
server_key: "5432c90c46034ea6672680a5f26c9b20"
server_key_iv: "8b6af40af1055d4bc8ed78e0794aa4ae"
server_key_version: "encryptionkey@0"
I20260808 10:43:59.260267 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "4d645fd1548d4f04a3d52948978c149a"
format_stamp: "Formatted at 2026-08-08 10:43:59 on dist-test-slave-ss4n"
server_key: "5432c90c46034ea6672680a5f26c9b20"
server_key_iv: "8b6af40af1055d4bc8ed78e0794aa4ae"
server_key_version: "encryptionkey@0"
I20260808 10:43:59.264036 24490 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:43:59.266522 24505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:43:59.267762 24490 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:43:59.267904 24490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "4d645fd1548d4f04a3d52948978c149a"
format_stamp: "Formatted at 2026-08-08 10:43:59 on dist-test-slave-ss4n"
server_key: "5432c90c46034ea6672680a5f26c9b20"
server_key_iv: "8b6af40af1055d4bc8ed78e0794aa4ae"
server_key_version: "encryptionkey@0"
I20260808 10:43:59.268019 24490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:43:59.280336 24490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:43:59.283800 24490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:43:59.284020 24490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:43:59.291162 24490 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34123
I20260808 10:43:59.291220 24558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34123 every 8 connection(s)
I20260808 10:43:59.292186 24490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:43:59.295043 24560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:43:59.297044 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24490
I20260808 10:43:59.297222 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:43:59.297554 1801 external_mini_cluster.cc:1428] Setting key 7e18e3266c29648c4d0caa8fd846b10a
I20260808 10:43:59.302066 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a: Bootstrap starting.
I20260808 10:43:59.304355 24560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a: Neither blocks nor log segments found. Creating new log.
I20260808 10:43:59.305099 24560 log.cc:824] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:43:59 dist-test-slave-ss4n krb5kdc[24476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:43:59.307344 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a: No bootstrap required, opened a new log
I20260808 10:43:59.311295 24560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d645fd1548d4f04a3d52948978c149a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34123 } }
I20260808 10:43:59.311625 24560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:43:59.311707 24560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d645fd1548d4f04a3d52948978c149a, State: Initialized, Role: FOLLOWER
I20260808 10:43:59.312206 24560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d645fd1548d4f04a3d52948978c149a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34123 } }
I20260808 10:43:59.312362 24560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:43:59.312449 24560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:43:59.312549 24560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:43:59.312758 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:59.299029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56039 (local address 127.0.0.1:34123)
0808 10:43:59.299508 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:59.299523 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:59.299561 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:43:59.300308 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:59.300326 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:59.300679 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:59.301028 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:59.301920 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:59.302993 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:59.303905 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:59.304387 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:59.304557 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:59.307540 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:59.307545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:59.307559 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:59.307586 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:59.309390 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:59.309981 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:59.309986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:59.309991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:59.310098 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:59.310451 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:59.310454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:59.310455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:59.310843 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:59.311015 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:59.311431 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 10:43:59.311814 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":151,"threads_started":1}
I20260808 10:43:59.313611 24560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d645fd1548d4f04a3d52948978c149a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34123 } }
I20260808 10:43:59.313974 24560 leader_election.cc:302] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d645fd1548d4f04a3d52948978c149a; no voters:
I20260808 10:43:59.314323 24560 leader_election.cc:288] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:43:59.314455 24565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:43:59.314742 24565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [term 1 LEADER]: Becoming Leader. State: Replica: 4d645fd1548d4f04a3d52948978c149a, State: Running, Role: LEADER
I20260808 10:43:59.315114 24565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d645fd1548d4f04a3d52948978c149a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34123 } }
I20260808 10:43:59.315680 24560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:43:59.316241 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:43:59.316385 24565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d645fd1548d4f04a3d52948978c149a. Latest consensus state: current_term: 1 leader_uuid: "4d645fd1548d4f04a3d52948978c149a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d645fd1548d4f04a3d52948978c149a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34123 } } }
I20260808 10:43:59.316493 24565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:59.317176 24567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d645fd1548d4f04a3d52948978c149a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d645fd1548d4f04a3d52948978c149a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34123 } } }
I20260808 10:43:59.317276 24567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a [sys.catalog]: This master's current role is: LEADER
I20260808 10:43:59.317567 24572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:43:59.321405 24572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:43:59.327714 24572 catalog_manager.cc:1347] Generated new cluster ID: 011c0fd95140487ca979669d87cf5b3c
I20260808 10:43:59.328079 24572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:43:59.330420 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:43:59.318871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:52791 (local address 127.0.0.1:34123)
0808 10:43:59.318972 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 10:43:59.318976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:43:59.319164 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:43:59.319243 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:43:59.319246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:43:59.319296 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:43:59.319385 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:43:59.320247 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:59.321011 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:59.322168 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:43:59.322520 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:43:59.322639 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:43:59.326562 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:43:59.326566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:43:59.326569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:43:59.326595 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:43:59.328455 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:59.329031 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:59.329035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:59.329037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:59.329119 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:43:59.329512 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:43:59.329515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:43:59.329517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:43:59.329741 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:43:59.329890 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:43:59.330179 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 10:43:59.330272 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:43:59.347030 24572 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:43:59.348364 24572 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:43:59.356967 24572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d645fd1548d4f04a3d52948978c149a: Generated new TSK 0
I20260808 10:43:59.357930 24572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:43:59.372534 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24490
W20260808 10:43:59.379225 24474 connection.cc:535] client connection to 127.0.0.1:34123 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34123): Connection reset by peer (error 104)
2026-08-08T10:43:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:43:59 dist-test-slave-ss4n krb5kdc[24593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:43:59 dist-test-slave-ss4n krb5kdc[24593](info): set up 2 sockets
Aug 08 10:43:59 dist-test-slave-ss4n krb5kdc[24593](info): commencing operation
krb5kdc: starting...
W20260808 10:44:00.453305 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:00 dist-test-slave-ss4n krb5kdc[24593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:44:00.589799 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:00.705260 24607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:00.705643 24607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:00.705729 24607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:00.709914 24607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:00.710007 24607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:00.710045 24607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:00.710103 24607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:00.710145 24607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:00.715531 24607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38843
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:00.716647 24607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:00.717721 24607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:00.726431 24613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:00.726431 24612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:00.726459 24615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:00.726786 24607 server_base.cc:1034] running on GCE node
I20260808 10:44:00.727373 24607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:00.728514 24607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:00.729699 24607 hybrid_clock.cc:648] HybridClock initialized: now 1786185840729684 us; error 35 us; skew 500 ppm
Aug 08 10:44:00 dist-test-slave-ss4n krb5kdc[24593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:00.732384 24607 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:44:00.733295 24607 webserver.cc:460] Webserver started at http://127.0.0.1:34881/ using document root <none> and password file <none>
I20260808 10:44:00.733914 24607 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:00.734009 24607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:00.734247 24607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:00.736263 24607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7"
format_stamp: "Formatted at 2026-08-08 10:44:00 on dist-test-slave-ss4n"
server_key: "3351f5f707f5c84d6700792a0bbb45fb"
server_key_iv: "58c352df8dd694188c38cc91452d1f2c"
server_key_version: "encryptionkey@0"
I20260808 10:44:00.736829 24607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7"
format_stamp: "Formatted at 2026-08-08 10:44:00 on dist-test-slave-ss4n"
server_key: "3351f5f707f5c84d6700792a0bbb45fb"
server_key_iv: "58c352df8dd694188c38cc91452d1f2c"
server_key_version: "encryptionkey@0"
I20260808 10:44:00.740475 24607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:44:00.742954 24622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:00.744163 24607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:00.744302 24607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7"
format_stamp: "Formatted at 2026-08-08 10:44:00 on dist-test-slave-ss4n"
server_key: "3351f5f707f5c84d6700792a0bbb45fb"
server_key_iv: "58c352df8dd694188c38cc91452d1f2c"
server_key_version: "encryptionkey@0"
I20260808 10:44:00.744418 24607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:00.761971 24607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:00.765027 24607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:00.765252 24607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:00.773041 24607 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38843
I20260808 10:44:00.773063 24677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38843 every 8 connection(s)
I20260808 10:44:00.774139 24607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:00.776815 24678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:00.777247 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24607
I20260808 10:44:00.777388 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:00.777781 1801 external_mini_cluster.cc:1428] Setting key 197bdfdd2ddfe2674d2a530021916fd1
I20260808 10:44:00.784672 24678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7: Bootstrap starting.
I20260808 10:44:00.787498 24678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:00.788367 24678 log.cc:824] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:00.788471 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:00.779236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:34077 (local address 127.0.0.1:38843)
0808 10:44:00.779869 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:00.779890 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:00.779933 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:44:00.780824 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:00.780848 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:00.781287 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:00.781676 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:00.782073 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:00.783616 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:00.784507 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:00.785178 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:00.785336 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:00.786459 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:00.786465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:00.786480 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:00.786967 (+ 487us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:00.787031 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:00.787060 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:00.787190 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:00.787411 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 10:44:00.787551 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":133,"threads_started":1}
I20260808 10:44:00.790498 24678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7: No bootstrap required, opened a new log
I20260808 10:44:00.791759 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:00.794831 24678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38843 } }
I20260808 10:44:00.795100 24678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:00.795178 24678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c41bf1fe4804ea4998c9488e7b5c7b7, State: Initialized, Role: FOLLOWER
I20260808 10:44:00.795637 24678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38843 } }
I20260808 10:44:00.795781 24678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:00.795850 24678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:00.795969 24678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:00.796973 24678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38843 } }
I20260808 10:44:00.797339 24678 leader_election.cc:302] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c41bf1fe4804ea4998c9488e7b5c7b7; no voters:
I20260808 10:44:00.797744 24678 leader_election.cc:288] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:00.797838 24688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:00.798110 24688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [term 1 LEADER]: Becoming Leader. State: Replica: 7c41bf1fe4804ea4998c9488e7b5c7b7, State: Running, Role: LEADER
I20260808 10:44:00.798501 24688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38843 } }
I20260808 10:44:00.798810 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:00.794076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:42575 (local address 127.0.0.1:38843)
0808 10:44:00.794203 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:00.794207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:00.794458 (+ 251us) server_negotiation.cc:407] Connection header received
0808 10:44:00.794546 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:00.794549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:00.794624 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:00.794706 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:00.795031 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:00.795804 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:00.796782 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:00.797146 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:00.797253 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:00.798282 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:00.798286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:00.798288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:00.798300 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:00.798327 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:00.798339 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:00.798398 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:00.798650 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 10:44:00.798684 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:44:00.799117 24678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:00.800190 24689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38843 } } }
I20260808 10:44:00.800330 24689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:00.800586 24689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c41bf1fe4804ea4998c9488e7b5c7b7. Latest consensus state: current_term: 1 leader_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c41bf1fe4804ea4998c9488e7b5c7b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38843 } } }
I20260808 10:44:00.800684 24689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:00.800797 24697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:00.803495 24697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:00.809383 24697 catalog_manager.cc:1347] Generated new cluster ID: b4f5c2d04f8a46c2964751ba52e49e2a
I20260808 10:44:00.809464 24697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:00.818195 24697 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:00.819144 24697 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:00.828385 24697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c41bf1fe4804ea4998c9488e7b5c7b7: Generated new TSK 0
I20260808 10:44:00.829375 24697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:00.834152 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24607
W20260808 10:44:00.841706 24591 connection.cc:535] client connection to 127.0.0.1:38843 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38843): Connection reset by peer (error 104)
2026-08-08T10:44:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:00 dist-test-slave-ss4n krb5kdc[24710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:00 dist-test-slave-ss4n krb5kdc[24710](info): set up 2 sockets
Aug 08 10:44:00 dist-test-slave-ss4n krb5kdc[24710](info): commencing operation
krb5kdc: starting...
W20260808 10:44:01.995811 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:02 dist-test-slave-ss4n krb5kdc[24710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:44:02.130395 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:02.240662 24723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:02.240917 24723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:02.240960 24723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:02.245615 24723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:02.245677 24723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:02.245697 24723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:02.245713 24723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:02.245767 24723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:02.251359 24723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:02.252457 24723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:02.253609 24723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:02.262380 24731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:02.262401 24728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:02.262566 24723 server_base.cc:1034] running on GCE node
W20260808 10:44:02.262408 24729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:02.263330 24723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:02.264509 24723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:02.265707 24723 hybrid_clock.cc:648] HybridClock initialized: now 1786185842265693 us; error 27 us; skew 500 ppm
Aug 08 10:44:02 dist-test-slave-ss4n krb5kdc[24710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:02.273733 24723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:44:02.274722 24723 webserver.cc:460] Webserver started at http://127.0.0.1:39623/ using document root <none> and password file <none>
I20260808 10:44:02.275293 24723 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:02.275383 24723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:02.275635 24723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:02.277769 24723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "be5919314d4c4a96b38ba727b0bd2a41"
format_stamp: "Formatted at 2026-08-08 10:44:02 on dist-test-slave-ss4n"
server_key: "d45f5af88f5853f6f99b76bf6c75bb98"
server_key_iv: "801827449153db9829ccf40bb66cc7ac"
server_key_version: "encryptionkey@0"
I20260808 10:44:02.278324 24723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "be5919314d4c4a96b38ba727b0bd2a41"
format_stamp: "Formatted at 2026-08-08 10:44:02 on dist-test-slave-ss4n"
server_key: "d45f5af88f5853f6f99b76bf6c75bb98"
server_key_iv: "801827449153db9829ccf40bb66cc7ac"
server_key_version: "encryptionkey@0"
I20260808 10:44:02.282025 24723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:02.284440 24738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:02.285681 24723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:02.285823 24723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "be5919314d4c4a96b38ba727b0bd2a41"
format_stamp: "Formatted at 2026-08-08 10:44:02 on dist-test-slave-ss4n"
server_key: "d45f5af88f5853f6f99b76bf6c75bb98"
server_key_iv: "801827449153db9829ccf40bb66cc7ac"
server_key_version: "encryptionkey@0"
I20260808 10:44:02.285939 24723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:02.290591 24723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:02.293437 24723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:02.293660 24723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:02.300688 24723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38761
I20260808 10:44:02.300710 24792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38761 every 8 connection(s)
I20260808 10:44:02.301690 24723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:02.304502 24793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:02.307164 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24723
I20260808 10:44:02.307278 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:02.307555 1801 external_mini_cluster.cc:1428] Setting key fe7570d2a57279dcd3b15c95465f91b2
I20260808 10:44:02.310453 24793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41: Bootstrap starting.
I20260808 10:44:02.312628 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:02.308968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:33111 (local address 127.0.0.1:38761)
0808 10:44:02.309411 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:02.309425 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:02.309459 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:44:02.310107 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:02.310126 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:02.310391 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:02.310725 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:02.311505 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:02.311509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:02.311520 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:02.311529 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:02.311581 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:02.311599 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:02.311751 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:02.311819 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:44:02.311970 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":139,"threads_started":1}
I20260808 10:44:02.312891 24793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:02.313730 24793 log.cc:824] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:02.315450 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:02.315877 24793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41: No bootstrap required, opened a new log
I20260808 10:44:02.319360 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:02.317675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:36405 (local address 127.0.0.1:38761)
0808 10:44:02.317792 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:02.317798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:02.317980 (+ 182us) server_negotiation.cc:407] Connection header received
0808 10:44:02.318116 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:02.318119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:02.318150 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:02.318240 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:02.318955 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:02.318958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:02.318961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:02.318971 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:02.318996 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:02.319009 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:02.319099 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:02.319192 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:44:02.319233 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:44:02.320474 24793 raft_consensus.cc:348] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5919314d4c4a96b38ba727b0bd2a41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38761 } }
I20260808 10:44:02.320693 24793 raft_consensus.cc:374] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:02.320727 24793 raft_consensus.cc:729] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be5919314d4c4a96b38ba727b0bd2a41, State: Initialized, Role: FOLLOWER
I20260808 10:44:02.321072 24793 consensus_queue.cc:260] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5919314d4c4a96b38ba727b0bd2a41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38761 } }
I20260808 10:44:02.321185 24793 raft_consensus.cc:388] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:02.321228 24793 raft_consensus.cc:482] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:02.321295 24793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:02.322207 24793 raft_consensus.cc:504] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5919314d4c4a96b38ba727b0bd2a41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38761 } }
I20260808 10:44:02.322489 24793 leader_election.cc:302] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be5919314d4c4a96b38ba727b0bd2a41; no voters:
I20260808 10:44:02.322731 24793 leader_election.cc:288] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:02.322847 24804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:02.323068 24804 raft_consensus.cc:686] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [term 1 LEADER]: Becoming Leader. State: Replica: be5919314d4c4a96b38ba727b0bd2a41, State: Running, Role: LEADER
I20260808 10:44:02.323429 24804 consensus_queue.cc:237] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5919314d4c4a96b38ba727b0bd2a41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38761 } }
I20260808 10:44:02.323972 24793 sys_catalog.cc:564] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:02.325150 24805 sys_catalog.cc:455] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be5919314d4c4a96b38ba727b0bd2a41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5919314d4c4a96b38ba727b0bd2a41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38761 } } }
I20260808 10:44:02.325139 24806 sys_catalog.cc:455] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be5919314d4c4a96b38ba727b0bd2a41. Latest consensus state: current_term: 1 leader_uuid: "be5919314d4c4a96b38ba727b0bd2a41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5919314d4c4a96b38ba727b0bd2a41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38761 } } }
I20260808 10:44:02.325284 24806 sys_catalog.cc:458] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:02.325284 24805 sys_catalog.cc:458] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:02.325881 24814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:02.328570 24814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:02.333999 24814 catalog_manager.cc:1347] Generated new cluster ID: a4391767d98c44958ad9ca149a68c7c3
I20260808 10:44:02.334069 24814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:02.346174 24814 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:02.346494 24814 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:02.352077 24814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be5919314d4c4a96b38ba727b0bd2a41: Generated new TSK 0
I20260808 10:44:02.352753 24814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:02.355562 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24723
2026-08-08T10:44:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:02 dist-test-slave-ss4n krb5kdc[24826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:02 dist-test-slave-ss4n krb5kdc[24826](info): set up 2 sockets
Aug 08 10:44:02 dist-test-slave-ss4n krb5kdc[24826](info): commencing operation
krb5kdc: starting...
W20260808 10:44:03.439819 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:03 dist-test-slave-ss4n krb5kdc[24826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:03.573828 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:41307
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:41307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:03.684297 24839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:03.684629 24839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:03.684763 24839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:03.688918 24839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:03.689019 24839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:03.689060 24839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:03.689119 24839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:03.689158 24839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:03.694485 24839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:41307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:41307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:03.695559 24839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:03.696563 24839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:03.705281 24847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:03.705319 24845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:03.705313 24839 server_base.cc:1034] running on GCE node
W20260808 10:44:03.705363 24844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:03.706161 24839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:03.707253 24839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:03.708422 24839 hybrid_clock.cc:648] HybridClock initialized: now 1786185843708484 us; error 159 us; skew 500 ppm
Aug 08 10:44:03 dist-test-slave-ss4n krb5kdc[24826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185843, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:03.714951 24839 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:03.715888 24839 webserver.cc:460] Webserver started at http://127.1.194.126:45361/ using document root <none> and password file <none>
I20260808 10:44:03.716460 24839 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:03.716523 24839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:03.716770 24839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:03.718859 24839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "7c6eaecf4deb440cbffe2854812e8b6c"
format_stamp: "Formatted at 2026-08-08 10:44:03 on dist-test-slave-ss4n"
server_key: "ffc8a8abd71a1014a0a6392619957486"
server_key_iv: "ca23722b4dc42d549eeaf58e81bf0c13"
server_key_version: "encryptionkey@0"
I20260808 10:44:03.719414 24839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "7c6eaecf4deb440cbffe2854812e8b6c"
format_stamp: "Formatted at 2026-08-08 10:44:03 on dist-test-slave-ss4n"
server_key: "ffc8a8abd71a1014a0a6392619957486"
server_key_iv: "ca23722b4dc42d549eeaf58e81bf0c13"
server_key_version: "encryptionkey@0"
I20260808 10:44:03.723141 24839 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:44:03.725411 24854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:03.726643 24839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:03.726783 24839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "7c6eaecf4deb440cbffe2854812e8b6c"
format_stamp: "Formatted at 2026-08-08 10:44:03 on dist-test-slave-ss4n"
server_key: "ffc8a8abd71a1014a0a6392619957486"
server_key_iv: "ca23722b4dc42d549eeaf58e81bf0c13"
server_key_version: "encryptionkey@0"
I20260808 10:44:03.726892 24839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:03.764178 24839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:03.767611 24839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:03.767930 24839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:03.775547 24839 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:41307
I20260808 10:44:03.775579 24908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:41307 every 8 connection(s)
I20260808 10:44:03.776505 24839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:03.779244 24909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:03.781275 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24839
I20260808 10:44:03.781420 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:03.781729 1801 external_mini_cluster.cc:1428] Setting key d5e28281fd303a3e8a8c130c33bf5eac
I20260808 10:44:03.785250 24909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c: Bootstrap starting.
I20260808 10:44:03.787770 24909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:03.788563 24909 log.cc:824] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:03.790813 24909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c: No bootstrap required, opened a new log
Aug 08 10:44:03 dist-test-slave-ss4n krb5kdc[24826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:44:03.795104 24909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41307 } }
I20260808 10:44:03.795401 24909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:03.795468 24909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6eaecf4deb440cbffe2854812e8b6c, State: Initialized, Role: FOLLOWER
I20260808 10:44:03.795990 24909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41307 } }
I20260808 10:44:03.796159 24909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:03.796221 24909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:03.796386 24909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:03.797643 24909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41307 } }
I20260808 10:44:03.797991 24909 leader_election.cc:302] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c6eaecf4deb440cbffe2854812e8b6c; no voters:
I20260808 10:44:03.798043 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:03.783167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50397 (local address 127.1.194.126:41307)
0808 10:44:03.783581 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:03.783597 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:03.783632 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:44:03.784302 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:03.784318 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:03.784693 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:03.784961 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:03.785917 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:03.786917 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:03.788105 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:03.788559 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:03.788738 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:03.791909 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:03.791915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:03.791931 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:03.791960 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:03.794598 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:03.795129 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:03.795137 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:03.795143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:03.795275 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:03.795696 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:03.795699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:03.795701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:03.796198 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:03.796431 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:03.796811 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 10:44:03.797030 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":137,"threads_started":1}
I20260808 10:44:03.798374 24909 leader_election.cc:288] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:03.798497 24914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:03.798751 24914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [term 1 LEADER]: Becoming Leader. State: Replica: 7c6eaecf4deb440cbffe2854812e8b6c, State: Running, Role: LEADER
I20260808 10:44:03.799126 24914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41307 } }
I20260808 10:44:03.799612 24909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:03.800742 24916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c6eaecf4deb440cbffe2854812e8b6c. Latest consensus state: current_term: 1 leader_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41307 } } }
I20260808 10:44:03.800870 24916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:03.801239 24915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6eaecf4deb440cbffe2854812e8b6c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 41307 } } }
I20260808 10:44:03.801317 24915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:03.801667 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:03.801769 24923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:03.804899 24923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:03.810597 24923 catalog_manager.cc:1347] Generated new cluster ID: 8b5011e6125f411daa6706388bcbfe13
I20260808 10:44:03.810756 24923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:03.813942 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:03.803961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39963 (local address 127.1.194.126:41307)
0808 10:44:03.804056 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:03.804061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:03.805278 (+ 1217us) server_negotiation.cc:407] Connection header received
0808 10:44:03.805400 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:03.805403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:03.805450 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:03.805576 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:03.806386 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:03.807211 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:03.808306 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:03.808686 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:03.808793 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:03.810528 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:03.810532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:03.810535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:03.810559 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:03.812104 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:03.812609 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:03.812612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:03.812614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:03.812695 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:03.813029 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:03.813033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:03.813034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:03.813210 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:03.813377 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:03.813684 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 10:44:03.813776 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:44:03.842432 24923 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:03.843729 24923 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:03.850208 24923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c6eaecf4deb440cbffe2854812e8b6c: Generated new TSK 0
I20260808 10:44:03.851003 24923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:03.859419 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24839
W20260808 10:44:03.866386 24824 connection.cc:535] client connection to 127.1.194.126:41307 recv error: Network error: failed to read from TLS socket (remote: 127.1.194.126:41307): Connection reset by peer (error 104)
2026-08-08T10:44:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:03 dist-test-slave-ss4n krb5kdc[24940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:03 dist-test-slave-ss4n krb5kdc[24940](info): set up 2 sockets
Aug 08 10:44:03 dist-test-slave-ss4n krb5kdc[24940](info): commencing operation
krb5kdc: starting...
W20260808 10:44:04.987217 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:05 dist-test-slave-ss4n krb5kdc[24940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:05.124678 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:43263
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:43263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:05.235141 24954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:05.235412 24954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:05.235455 24954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:05.239373 24954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:05.239425 24954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:05.239444 24954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:05.239460 24954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:05.239475 24954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:05.244658 24954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:43263
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:43263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:05.245685 24954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:05.246631 24954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:05.255235 24960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:05.255249 24959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:05.255225 24962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:05.255460 24954 server_base.cc:1034] running on GCE node
I20260808 10:44:05.255990 24954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:05.257081 24954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:05.258244 24954 hybrid_clock.cc:648] HybridClock initialized: now 1786185845258224 us; error 31 us; skew 500 ppm
Aug 08 10:44:05 dist-test-slave-ss4n krb5kdc[24940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185845, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:05.261027 24954 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:05.262064 24954 webserver.cc:460] Webserver started at http://127.1.194.126:40341/ using document root <none> and password file <none>
I20260808 10:44:05.262631 24954 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:05.262693 24954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:05.262908 24954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:05.264940 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "4b874e8fa5864672b1e0b0b6a25f8d76"
format_stamp: "Formatted at 2026-08-08 10:44:05 on dist-test-slave-ss4n"
server_key: "c042fb2c8a40e319b710cef8f9f4ecea"
server_key_iv: "c039159aae2474a139224c23a46c4855"
server_key_version: "encryptionkey@0"
I20260808 10:44:05.265561 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "4b874e8fa5864672b1e0b0b6a25f8d76"
format_stamp: "Formatted at 2026-08-08 10:44:05 on dist-test-slave-ss4n"
server_key: "c042fb2c8a40e319b710cef8f9f4ecea"
server_key_iv: "c039159aae2474a139224c23a46c4855"
server_key_version: "encryptionkey@0"
I20260808 10:44:05.269215 24954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:05.271692 24969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:05.272920 24954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:05.273058 24954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "4b874e8fa5864672b1e0b0b6a25f8d76"
format_stamp: "Formatted at 2026-08-08 10:44:05 on dist-test-slave-ss4n"
server_key: "c042fb2c8a40e319b710cef8f9f4ecea"
server_key_iv: "c039159aae2474a139224c23a46c4855"
server_key_version: "encryptionkey@0"
I20260808 10:44:05.273187 24954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:05.298295 24954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:05.301719 24954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:05.301928 24954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:05.309554 24954 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:43263
I20260808 10:44:05.309583 25024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:43263 every 8 connection(s)
I20260808 10:44:05.310532 24954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:05.311946 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 24954
I20260808 10:44:05.312101 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:05.312425 1801 external_mini_cluster.cc:1428] Setting key ea68d106a06ac9339d3ae4d2d3dec6c0
I20260808 10:44:05.313758 25025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:05.319811 25025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76: Bootstrap starting.
I20260808 10:44:05.321424 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:05.313945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54341 (local address 127.1.194.126:43263)
0808 10:44:05.314396 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:05.314411 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:05.314446 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:44:05.315089 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:05.315104 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:05.315430 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:05.315766 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:05.316167 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:05.317139 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:05.318100 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:05.318574 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:05.318739 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:05.319722 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:05.319727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:05.319741 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:05.320203 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:05.320260 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:05.320283 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:05.320432 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:05.320586 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 10:44:05.320737 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":154,"threads_started":1}
I20260808 10:44:05.322600 25025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:05.323501 25025 log.cc:824] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:05.324775 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:05.325781 25025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76: No bootstrap required, opened a new log
I20260808 10:44:05.330118 25025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43263 } }
I20260808 10:44:05.330410 25025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:05.330502 25025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b874e8fa5864672b1e0b0b6a25f8d76, State: Initialized, Role: FOLLOWER
I20260808 10:44:05.331048 25025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43263 } }
I20260808 10:44:05.331238 25025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:05.331321 25025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:05.331450 25025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:05.332162 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:05.326953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47793 (local address 127.1.194.126:43263)
0808 10:44:05.327081 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:05.327094 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:05.327314 (+ 220us) server_negotiation.cc:407] Connection header received
0808 10:44:05.327463 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:05.327466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:05.327508 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:05.327592 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:05.327949 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:05.328899 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:05.330187 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:05.330550 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:05.330666 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:05.331669 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:05.331676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:05.331683 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:05.331701 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:05.331736 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:05.331754 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:05.331859 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:05.331980 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 10:44:05.332021 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:44:05.332779 25025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43263 } }
I20260808 10:44:05.333211 25025 leader_election.cc:302] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b874e8fa5864672b1e0b0b6a25f8d76; no voters:
I20260808 10:44:05.333618 25025 leader_election.cc:288] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:05.333741 25035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:05.333999 25035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [term 1 LEADER]: Becoming Leader. State: Replica: 4b874e8fa5864672b1e0b0b6a25f8d76, State: Running, Role: LEADER
I20260808 10:44:05.334326 25035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43263 } }
I20260808 10:44:05.334928 25025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:05.336114 25037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b874e8fa5864672b1e0b0b6a25f8d76. Latest consensus state: current_term: 1 leader_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43263 } } }
I20260808 10:44:05.336118 25036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b874e8fa5864672b1e0b0b6a25f8d76" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43263 } } }
I20260808 10:44:05.336253 25037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:05.336280 25036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:05.337280 25045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:05.340085 25045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:05.346004 25045 catalog_manager.cc:1347] Generated new cluster ID: 343ce7f99cd946e6b24c2224a811d95e
I20260808 10:44:05.346088 25045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:05.360453 25045 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:05.361866 25045 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:05.371657 25045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b874e8fa5864672b1e0b0b6a25f8d76: Generated new TSK 0
I20260808 10:44:05.372499 25045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:05.381371 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 24954
W20260808 10:44:05.388590 24938 connection.cc:535] client connection to 127.1.194.126:43263 recv error: Network error: failed to read from TLS socket (remote: 127.1.194.126:43263): Connection reset by peer (error 104)
2026-08-08T10:44:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:05 dist-test-slave-ss4n krb5kdc[25057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:05 dist-test-slave-ss4n krb5kdc[25057](info): set up 2 sockets
Aug 08 10:44:05 dist-test-slave-ss4n krb5kdc[25057](info): commencing operation
krb5kdc: starting...
W20260808 10:44:06.473367 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:06 dist-test-slave-ss4n krb5kdc[25057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:06.613675 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:43011
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:43011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:06.730392 25070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:06.730656 25070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:06.730698 25070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:06.734661 25070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:06.734717 25070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:06.734736 25070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:06.734753 25070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:06.734769 25070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:06.739861 25070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:43011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:43011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:06.740906 25070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:06.741936 25070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:06.750455 25078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:06.750409 25076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:06.750483 25075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:06.750833 25070 server_base.cc:1034] running on GCE node
I20260808 10:44:06.751569 25070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:06.752663 25070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:06.753852 25070 hybrid_clock.cc:648] HybridClock initialized: now 1786185846753838 us; error 30 us; skew 500 ppm
Aug 08 10:44:06 dist-test-slave-ss4n krb5kdc[25057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:06.756691 25070 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:06.757597 25070 webserver.cc:460] Webserver started at http://127.1.194.126:42299/ using document root <none> and password file <none>
I20260808 10:44:06.758158 25070 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:06.758252 25070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:06.758482 25070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:06.760582 25070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "feb45f0c9446412c887e6cb110b2bc93"
format_stamp: "Formatted at 2026-08-08 10:44:06 on dist-test-slave-ss4n"
server_key: "8f233d7c755524d8d865b49af4991cba"
server_key_iv: "35e28e3246dc8610298f1702bc52bc85"
server_key_version: "encryptionkey@0"
I20260808 10:44:06.761170 25070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "feb45f0c9446412c887e6cb110b2bc93"
format_stamp: "Formatted at 2026-08-08 10:44:06 on dist-test-slave-ss4n"
server_key: "8f233d7c755524d8d865b49af4991cba"
server_key_iv: "35e28e3246dc8610298f1702bc52bc85"
server_key_version: "encryptionkey@0"
I20260808 10:44:06.764863 25070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:44:06.767288 25086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:06.768512 25070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:44:06.768657 25070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "feb45f0c9446412c887e6cb110b2bc93"
format_stamp: "Formatted at 2026-08-08 10:44:06 on dist-test-slave-ss4n"
server_key: "8f233d7c755524d8d865b49af4991cba"
server_key_iv: "35e28e3246dc8610298f1702bc52bc85"
server_key_version: "encryptionkey@0"
I20260808 10:44:06.768769 25070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:06.773340 25070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:06.776446 25070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:06.776656 25070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:06.784027 25070 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:43011
I20260808 10:44:06.784092 25140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:43011 every 8 connection(s)
I20260808 10:44:06.785048 25070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:06.787919 25141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:06.791285 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25070
I20260808 10:44:06.791440 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:06.791745 1801 external_mini_cluster.cc:1428] Setting key a50917565f7f0ef2f24f9eb0deb33690
I20260808 10:44:06.794278 25141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93: Bootstrap starting.
I20260808 10:44:06.797116 25141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:06.797425 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:06.793191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53099 (local address 127.1.194.126:43011)
0808 10:44:06.793648 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:06.793664 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:06.793700 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:44:06.794349 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:06.794365 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:06.794754 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:06.795047 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:06.795982 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:06.795987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:06.795999 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:06.796010 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:06.796065 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:06.796084 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:06.796203 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:06.796350 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 10:44:06.796581 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":137,"threads_started":1}
I20260808 10:44:06.798151 25141 log.cc:824] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:06.800511 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:06.800595 25141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93: No bootstrap required, opened a new log
I20260808 10:44:06.805202 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:06.803133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60493 (local address 127.1.194.126:43011)
0808 10:44:06.803288 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:06.803293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:06.803821 (+ 528us) server_negotiation.cc:407] Connection header received
0808 10:44:06.803936 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:06.803939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:06.803994 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:06.804062 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:06.804812 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:06.804815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:06.804818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:06.804829 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:06.804857 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:06.804876 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:06.804967 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:06.805042 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:44:06.805080 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:44:06.805642 25141 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb45f0c9446412c887e6cb110b2bc93" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43011 } }
I20260808 10:44:06.805908 25141 raft_consensus.cc:374] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:06.806015 25141 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb45f0c9446412c887e6cb110b2bc93, State: Initialized, Role: FOLLOWER
I20260808 10:44:06.806548 25141 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb45f0c9446412c887e6cb110b2bc93" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43011 } }
I20260808 10:44:06.806706 25141 raft_consensus.cc:388] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:06.806783 25141 raft_consensus.cc:482] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:06.806893 25141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:06.807894 25141 raft_consensus.cc:504] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb45f0c9446412c887e6cb110b2bc93" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43011 } }
I20260808 10:44:06.808251 25141 leader_election.cc:302] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feb45f0c9446412c887e6cb110b2bc93; no voters:
I20260808 10:44:06.808584 25141 leader_election.cc:288] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:06.808703 25151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:06.808944 25151 raft_consensus.cc:686] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [term 1 LEADER]: Becoming Leader. State: Replica: feb45f0c9446412c887e6cb110b2bc93, State: Running, Role: LEADER
I20260808 10:44:06.809322 25151 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb45f0c9446412c887e6cb110b2bc93" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43011 } }
I20260808 10:44:06.809981 25141 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:06.810992 25154 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb45f0c9446412c887e6cb110b2bc93. Latest consensus state: current_term: 1 leader_uuid: "feb45f0c9446412c887e6cb110b2bc93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb45f0c9446412c887e6cb110b2bc93" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43011 } } }
I20260808 10:44:06.811120 25154 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:06.811498 25161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:06.812729 25152 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feb45f0c9446412c887e6cb110b2bc93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb45f0c9446412c887e6cb110b2bc93" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 43011 } } }
I20260808 10:44:06.812896 25152 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:06.814908 25161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:06.820443 25161 catalog_manager.cc:1347] Generated new cluster ID: bb96a14c65de4d41b2f73642b52d014f
I20260808 10:44:06.820516 25161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:06.842906 25161 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:06.843252 25161 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:06.857188 25161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feb45f0c9446412c887e6cb110b2bc93: Generated new TSK 0
I20260808 10:44:06.857957 25161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:06.864022 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25070
2026-08-08T10:44:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:06 dist-test-slave-ss4n krb5kdc[25174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:06 dist-test-slave-ss4n krb5kdc[25174](info): set up 2 sockets
Aug 08 10:44:06 dist-test-slave-ss4n krb5kdc[25174](info): commencing operation
krb5kdc: starting...
W20260808 10:44:07.991861 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:08 dist-test-slave-ss4n krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:08.129803 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42633
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:08.241366 25188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:08.241696 25188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:08.241740 25188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:08.245744 25188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:08.245811 25188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:08.245831 25188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:08.245848 25188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:08.245863 25188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:08.251147 25188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:08.252290 25188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:08.253283 25188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:08.262231 25196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:08.262322 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:08.262465 25188 server_base.cc:1034] running on GCE node
W20260808 10:44:08.262316 25194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:08.263163 25188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:08.264417 25188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:08.265619 25188 hybrid_clock.cc:648] HybridClock initialized: now 1786185848265542 us; error 31 us; skew 500 ppm
Aug 08 10:44:08 dist-test-slave-ss4n krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185848, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:08.272356 25188 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:08.273353 25188 webserver.cc:460] Webserver started at http://127.1.194.126:42507/ using document root <none> and password file <none>
I20260808 10:44:08.274281 25188 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:08.274354 25188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:08.274542 25188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:08.276701 25188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "07fceed098aa426a92c91303f090a81c"
format_stamp: "Formatted at 2026-08-08 10:44:08 on dist-test-slave-ss4n"
server_key: "a4f1868349db5e0bea18f9ef609185e2"
server_key_iv: "26a3b6e5c42771050445d02929ec9935"
server_key_version: "encryptionkey@0"
I20260808 10:44:08.277225 25188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "07fceed098aa426a92c91303f090a81c"
format_stamp: "Formatted at 2026-08-08 10:44:08 on dist-test-slave-ss4n"
server_key: "a4f1868349db5e0bea18f9ef609185e2"
server_key_iv: "26a3b6e5c42771050445d02929ec9935"
server_key_version: "encryptionkey@0"
I20260808 10:44:08.280978 25188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:08.283722 25203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:08.285055 25188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:08.285164 25188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "07fceed098aa426a92c91303f090a81c"
format_stamp: "Formatted at 2026-08-08 10:44:08 on dist-test-slave-ss4n"
server_key: "a4f1868349db5e0bea18f9ef609185e2"
server_key_iv: "26a3b6e5c42771050445d02929ec9935"
server_key_version: "encryptionkey@0"
I20260808 10:44:08.285315 25188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:08.297336 25188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:08.300906 25188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:08.301141 25188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:08.309088 25188 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:42633
I20260808 10:44:08.309115 25256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:42633 every 8 connection(s)
I20260808 10:44:08.310137 25188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:08.313171 25257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:08.316473 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25188
I20260808 10:44:08.316672 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:08.316941 1801 external_mini_cluster.cc:1428] Setting key 8edbaca963f17421c032d3c54abbafc8
I20260808 10:44:08.320322 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c: Bootstrap starting.
I20260808 10:44:08.323446 25257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:08.324388 25257 log.cc:824] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:08.326500 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c: No bootstrap required, opened a new log
Aug 08 10:44:08 dist-test-slave-ss4n krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:44:08.330621 25257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fceed098aa426a92c91303f090a81c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42633 } }
I20260808 10:44:08.330842 25257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:08.330915 25257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fceed098aa426a92c91303f090a81c, State: Initialized, Role: FOLLOWER
I20260808 10:44:08.331451 25257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fceed098aa426a92c91303f090a81c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42633 } }
I20260808 10:44:08.331607 25257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:08.331694 25257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:08.331813 25257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:08.332772 25257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fceed098aa426a92c91303f090a81c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42633 } }
I20260808 10:44:08.333230 25257 leader_election.cc:302] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07fceed098aa426a92c91303f090a81c; no voters:
I20260808 10:44:08.333633 25257 leader_election.cc:288] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:08.333742 25262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:08.333966 25262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [term 1 LEADER]: Becoming Leader. State: Replica: 07fceed098aa426a92c91303f090a81c, State: Running, Role: LEADER
I20260808 10:44:08.333990 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:08.318464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55631 (local address 127.1.194.126:42633)
0808 10:44:08.318956 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:08.318979 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:08.319028 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:44:08.319943 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:08.319968 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:08.320450 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:08.320863 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:08.321695 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:08.323365 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:08.324373 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:08.325089 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:08.325294 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:08.328494 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:08.328500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:08.328514 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:08.328546 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:08.330493 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:08.331014 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:08.331021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:08.331031 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:08.331146 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:08.331445 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:08.331449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:08.331451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:08.332070 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:08.332270 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:08.332604 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:44:08.332894 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":130,"threads_started":1}
I20260808 10:44:08.334705 25262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fceed098aa426a92c91303f090a81c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42633 } }
I20260808 10:44:08.334878 25257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:08.337051 25264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07fceed098aa426a92c91303f090a81c. Latest consensus state: current_term: 1 leader_uuid: "07fceed098aa426a92c91303f090a81c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fceed098aa426a92c91303f090a81c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42633 } } }
I20260808 10:44:08.337448 25263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07fceed098aa426a92c91303f090a81c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fceed098aa426a92c91303f090a81c" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42633 } } }
I20260808 10:44:08.337623 25264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:08.337623 25263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:08.337672 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 10:44:08.340113 25277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:08.340180 25277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:08.340265 25279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:08.341265 25279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:08.347680 25279 catalog_manager.cc:1347] Generated new cluster ID: 4ba6c9779ebc4cd6b0075f1167ea1b0e
I20260808 10:44:08.347748 25279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:08.349965 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:08.340662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51191 (local address 127.1.194.126:42633)
0808 10:44:08.340768 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:08.340773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:08.341017 (+ 244us) server_negotiation.cc:407] Connection header received
0808 10:44:08.341137 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:08.341141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:08.341187 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:08.341276 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:08.342203 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:08.343019 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:08.344197 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:08.344519 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:08.344626 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:08.346370 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:08.346374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:08.346377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:08.346407 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:08.348171 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:08.348660 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:08.348666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:08.348668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:08.348755 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:08.349031 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:08.349034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:08.349036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:08.349254 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:08.349387 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:08.349704 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 10:44:08.349815 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:44:08.354152 25279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:08.355439 25279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:08.369443 25279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07fceed098aa426a92c91303f090a81c: Generated new TSK 0
I20260808 10:44:08.370312 25279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:08.378258 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25188
2026-08-08T10:44:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:08 dist-test-slave-ss4n krb5kdc[25290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:08 dist-test-slave-ss4n krb5kdc[25290](info): set up 2 sockets
Aug 08 10:44:08 dist-test-slave-ss4n krb5kdc[25290](info): commencing operation
krb5kdc: starting...
W20260808 10:44:09.546401 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:09 dist-test-slave-ss4n krb5kdc[25290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:09.687390 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45867
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:45867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:09.797323 25303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:09.797725 25303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:09.797813 25303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:09.802107 25303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:09.802198 25303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:09.802237 25303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:09.802294 25303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:09.802335 25303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:09.807610 25303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:45867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:09.808708 25303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:09.809759 25303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:09.819468 25310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:09.819550 25309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:09.819558 25312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:09.819841 25303 server_base.cc:1034] running on GCE node
I20260808 10:44:09.820420 25303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:09.821547 25303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:09.822742 25303 hybrid_clock.cc:648] HybridClock initialized: now 1786185849822722 us; error 43 us; skew 500 ppm
Aug 08 10:44:09 dist-test-slave-ss4n krb5kdc[25290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:09.825541 25303 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:09.826383 25303 webserver.cc:460] Webserver started at http://127.1.194.126:42981/ using document root <none> and password file <none>
I20260808 10:44:09.826925 25303 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:09.827021 25303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:09.827267 25303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:09.829248 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "64488ea2425348c4b53bc0d632823296"
format_stamp: "Formatted at 2026-08-08 10:44:09 on dist-test-slave-ss4n"
server_key: "99fbc46af9c6b905401e14f71c24c1c8"
server_key_iv: "affb4921074c578642f79beec61fe5d2"
server_key_version: "encryptionkey@0"
I20260808 10:44:09.829860 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "64488ea2425348c4b53bc0d632823296"
format_stamp: "Formatted at 2026-08-08 10:44:09 on dist-test-slave-ss4n"
server_key: "99fbc46af9c6b905401e14f71c24c1c8"
server_key_iv: "affb4921074c578642f79beec61fe5d2"
server_key_version: "encryptionkey@0"
I20260808 10:44:09.833374 25303 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:09.835680 25319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:09.836886 25303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:09.837059 25303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "64488ea2425348c4b53bc0d632823296"
format_stamp: "Formatted at 2026-08-08 10:44:09 on dist-test-slave-ss4n"
server_key: "99fbc46af9c6b905401e14f71c24c1c8"
server_key_iv: "affb4921074c578642f79beec61fe5d2"
server_key_version: "encryptionkey@0"
I20260808 10:44:09.837174 25303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:09.872009 25303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:09.875696 25303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:09.875921 25303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:09.883762 25303 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:45867
I20260808 10:44:09.883790 25374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:45867 every 8 connection(s)
I20260808 10:44:09.884738 25303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:09.885084 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25303
I20260808 10:44:09.885249 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:09.885645 1801 external_mini_cluster.cc:1428] Setting key b3d1ee40d3ec932f6a343edd360eebe2
I20260808 10:44:09.887951 25375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:09.893342 25375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296: Bootstrap starting.
I20260808 10:44:09.894915 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:09.887090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37937 (local address 127.1.194.126:45867)
0808 10:44:09.887589 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:09.887605 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:09.887642 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:44:09.888363 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:09.888381 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:09.888727 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:09.889079 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:09.889554 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:09.890550 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:09.891473 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:09.891946 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:09.892141 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:09.893089 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:09.893094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:09.893106 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:09.893599 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:09.893672 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:09.893696 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:09.893807 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:09.893926 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 10:44:09.894137 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":136,"threads_started":1}
I20260808 10:44:09.895844 25375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:09.896600 25375 log.cc:824] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:09.897972 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:09.898640 25375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296: No bootstrap required, opened a new log
I20260808 10:44:09.902283 25375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64488ea2425348c4b53bc0d632823296" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45867 } }
I20260808 10:44:09.902552 25375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:09.902607 25375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64488ea2425348c4b53bc0d632823296, State: Initialized, Role: FOLLOWER
I20260808 10:44:09.903065 25375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64488ea2425348c4b53bc0d632823296" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45867 } }
I20260808 10:44:09.903226 25375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:09.903282 25375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:09.903379 25375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:09.904547 25375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64488ea2425348c4b53bc0d632823296" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45867 } }
I20260808 10:44:09.904959 25375 leader_election.cc:302] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64488ea2425348c4b53bc0d632823296; no voters:
I20260808 10:44:09.905279 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:09.900210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40249 (local address 127.1.194.126:45867)
0808 10:44:09.900352 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:09.900356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:09.900534 (+ 178us) server_negotiation.cc:407] Connection header received
0808 10:44:09.900678 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:09.900680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:09.900723 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:09.900793 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:09.901097 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:09.902366 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:09.903496 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:09.903772 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:09.903863 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:09.904754 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:09.904758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:09.904761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:09.904776 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:09.904812 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:09.904825 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:09.904902 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:09.905031 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:44:09.905111 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:44:09.905341 25375 leader_election.cc:288] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:09.905409 25386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:09.905691 25386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [term 1 LEADER]: Becoming Leader. State: Replica: 64488ea2425348c4b53bc0d632823296, State: Running, Role: LEADER
I20260808 10:44:09.906018 25386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64488ea2425348c4b53bc0d632823296" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45867 } }
I20260808 10:44:09.906646 25375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:09.907181 25386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64488ea2425348c4b53bc0d632823296. Latest consensus state: current_term: 1 leader_uuid: "64488ea2425348c4b53bc0d632823296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64488ea2425348c4b53bc0d632823296" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45867 } } }
I20260808 10:44:09.907366 25386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:09.907685 25387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64488ea2425348c4b53bc0d632823296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64488ea2425348c4b53bc0d632823296" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 45867 } } }
I20260808 10:44:09.907807 25393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:09.907847 25387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:09.910740 25393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:09.918978 25393 catalog_manager.cc:1347] Generated new cluster ID: 0657ebce1d1148f19536e6101b0df130
I20260808 10:44:09.919085 25393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:09.933030 25393 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:09.934327 25393 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:09.942807 25393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64488ea2425348c4b53bc0d632823296: Generated new TSK 0
I20260808 10:44:09.943666 25393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:09.951371 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25303
2026-08-08T10:44:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:10 dist-test-slave-ss4n krb5kdc[25407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:10 dist-test-slave-ss4n krb5kdc[25407](info): set up 2 sockets
Aug 08 10:44:10 dist-test-slave-ss4n krb5kdc[25407](info): commencing operation
krb5kdc: starting...
W20260808 10:44:11.075711 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:11 dist-test-slave-ss4n krb5kdc[25407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:11.209239 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:35187
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:35187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:11.323146 25421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:11.323479 25421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:11.323578 25421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:11.327811 25421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:11.327914 25421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:11.327952 25421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:11.328012 25421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:11.328055 25421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:11.333472 25421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:35187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:35187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:11.334623 25421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:11.335640 25421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:11.344282 25429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:11.344413 25427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:11.344357 25421 server_base.cc:1034] running on GCE node
W20260808 10:44:11.344564 25426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:11.345182 25421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:11.346448 25421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:11.347641 25421 hybrid_clock.cc:648] HybridClock initialized: now 1786185851347612 us; error 41 us; skew 500 ppm
Aug 08 10:44:11 dist-test-slave-ss4n krb5kdc[25407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:11.350792 25421 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:11.351706 25421 webserver.cc:460] Webserver started at http://127.1.194.126:34771/ using document root <none> and password file <none>
I20260808 10:44:11.352254 25421 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:11.352316 25421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:11.352502 25421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:11.354676 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "64fc694d72a74b00a319e0cc8c0f6080"
format_stamp: "Formatted at 2026-08-08 10:44:11 on dist-test-slave-ss4n"
server_key: "64e126ea3c7b8e3ba10db2486e1616fd"
server_key_iv: "f400c702012da989ebf61f5e546b76d9"
server_key_version: "encryptionkey@0"
I20260808 10:44:11.355211 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "64fc694d72a74b00a319e0cc8c0f6080"
format_stamp: "Formatted at 2026-08-08 10:44:11 on dist-test-slave-ss4n"
server_key: "64e126ea3c7b8e3ba10db2486e1616fd"
server_key_iv: "f400c702012da989ebf61f5e546b76d9"
server_key_version: "encryptionkey@0"
I20260808 10:44:11.359229 25421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:11.361939 25438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:11.363257 25421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:44:11.363402 25421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "64fc694d72a74b00a319e0cc8c0f6080"
format_stamp: "Formatted at 2026-08-08 10:44:11 on dist-test-slave-ss4n"
server_key: "64e126ea3c7b8e3ba10db2486e1616fd"
server_key_iv: "f400c702012da989ebf61f5e546b76d9"
server_key_version: "encryptionkey@0"
I20260808 10:44:11.363511 25421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:11.368358 25421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:11.371707 25421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:11.372016 25421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:11.379689 25421 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:35187
I20260808 10:44:11.379719 25491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:35187 every 8 connection(s)
I20260808 10:44:11.380643 25421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:11.383430 25492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:11.386709 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25421
I20260808 10:44:11.386904 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:11.387176 1801 external_mini_cluster.cc:1428] Setting key 4ecb0cc01651a4118b279862443c3cd7
I20260808 10:44:11.389420 25492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080: Bootstrap starting.
I20260808 10:44:11.391767 25492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:11.392573 25492 log.cc:824] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:11.392611 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:11.388654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57741 (local address 127.1.194.126:35187)
0808 10:44:11.389116 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:11.389131 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:11.389167 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:44:11.389911 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:11.389931 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:11.390221 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:11.390555 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:11.391400 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:11.391405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:11.391416 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:11.391426 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:11.391478 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:11.391498 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:11.391605 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:11.391753 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 10:44:11.391905 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":129,"threads_started":1}
I20260808 10:44:11.394723 25492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080: No bootstrap required, opened a new log
I20260808 10:44:11.395750 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:11.398749 25492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fc694d72a74b00a319e0cc8c0f6080" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35187 } }
I20260808 10:44:11.399050 25492 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:11.399153 25492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64fc694d72a74b00a319e0cc8c0f6080, State: Initialized, Role: FOLLOWER
I20260808 10:44:11.399636 25492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fc694d72a74b00a319e0cc8c0f6080" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35187 } }
I20260808 10:44:11.399806 25492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:11.399931 25492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:11.400064 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:11.398170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60247 (local address 127.1.194.126:35187)
0808 10:44:11.398302 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:11.398307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:11.398510 (+ 203us) server_negotiation.cc:407] Connection header received
0808 10:44:11.398620 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:11.398622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:11.398653 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:11.398720 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:11.399593 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:11.399598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:11.399604 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:11.399618 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:11.399659 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:11.399670 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:11.399748 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:11.399873 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 10:44:11.399914 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:44:11.400085 25492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:11.401077 25492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fc694d72a74b00a319e0cc8c0f6080" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35187 } }
I20260808 10:44:11.401423 25492 leader_election.cc:302] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64fc694d72a74b00a319e0cc8c0f6080; no voters:
I20260808 10:44:11.401816 25492 leader_election.cc:288] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:11.401883 25503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:11.402240 25503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [term 1 LEADER]: Becoming Leader. State: Replica: 64fc694d72a74b00a319e0cc8c0f6080, State: Running, Role: LEADER
I20260808 10:44:11.402565 25503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fc694d72a74b00a319e0cc8c0f6080" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35187 } }
I20260808 10:44:11.403142 25492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:11.404666 25504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64fc694d72a74b00a319e0cc8c0f6080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fc694d72a74b00a319e0cc8c0f6080" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35187 } } }
I20260808 10:44:11.404699 25505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64fc694d72a74b00a319e0cc8c0f6080. Latest consensus state: current_term: 1 leader_uuid: "64fc694d72a74b00a319e0cc8c0f6080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fc694d72a74b00a319e0cc8c0f6080" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 35187 } } }
I20260808 10:44:11.404863 25505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:11.405134 25504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:11.405205 25516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:11.408401 25516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:11.413856 25516 catalog_manager.cc:1347] Generated new cluster ID: ad6271fb44b64dcfb1de31a7319ee3f0
I20260808 10:44:11.413936 25516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:11.425930 25516 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:11.426259 25516 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:11.442293 25516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64fc694d72a74b00a319e0cc8c0f6080: Generated new TSK 0
I20260808 10:44:11.443150 25516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:11.456089 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25421
2026-08-08T10:44:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1506 ms)
[----------] 18 tests from AuthTokenIssuingTest (27068 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:11 dist-test-slave-ss4n krb5kdc[25525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:11 dist-test-slave-ss4n krb5kdc[25525](info): set up 2 sockets
Aug 08 10:44:11 dist-test-slave-ss4n krb5kdc[25525](info): commencing operation
krb5kdc: starting...
W20260808 10:44:12.564033 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:12 dist-test-slave-ss4n krb5kdc[25525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:12.696861 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42681
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42681,127.1.194.125:44929,127.1.194.124:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:12.811028 25539 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:44:12.811316 25539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:12.811340 25539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:12.811424 25539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:12.815439 25539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:12.815496 25539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:12.815516 25539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:12.815533 25539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:12.815548 25539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:12.820706 25539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42681,127.1.194.125:44929,127.1.194.124:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:12.821741 25539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:12.822818 25539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:44:12.833546 25539 server_base.cc:1034] running on GCE node
W20260808 10:44:12.833534 25545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:12.833446 25548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:12.833469 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:12.834405 25539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:12.835506 25539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:12.836681 25539 hybrid_clock.cc:648] HybridClock initialized: now 1786185852836653 us; error 41 us; skew 500 ppm
Aug 08 10:44:12 dist-test-slave-ss4n krb5kdc[25525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:12.839478 25539 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:12.840288 25539 webserver.cc:460] Webserver started at http://127.1.194.126:32869/ using document root <none> and password file <none>
I20260808 10:44:12.840796 25539 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:12.840855 25539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:12.841053 25539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:12.843123 25539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "841a1d698e694bcc8c2cbfa538ed0df8"
format_stamp: "Formatted at 2026-08-08 10:44:12 on dist-test-slave-ss4n"
server_key: "eec7cb097d219f8509f8f84f03eb7ab7"
server_key_iv: "4f33b43f3394bcbf24b0b0672df304a4"
server_key_version: "encryptionkey@0"
I20260808 10:44:12.843700 25539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "841a1d698e694bcc8c2cbfa538ed0df8"
format_stamp: "Formatted at 2026-08-08 10:44:12 on dist-test-slave-ss4n"
server_key: "eec7cb097d219f8509f8f84f03eb7ab7"
server_key_iv: "4f33b43f3394bcbf24b0b0672df304a4"
server_key_version: "encryptionkey@0"
I20260808 10:44:12.847247 25539 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 10:44:12.849617 25555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:12.850819 25539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:12.850961 25539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "841a1d698e694bcc8c2cbfa538ed0df8"
format_stamp: "Formatted at 2026-08-08 10:44:12 on dist-test-slave-ss4n"
server_key: "eec7cb097d219f8509f8f84f03eb7ab7"
server_key_iv: "4f33b43f3394bcbf24b0b0672df304a4"
server_key_version: "encryptionkey@0"
I20260808 10:44:12.851074 25539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:12.865422 25539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:12.868829 25539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:12.869041 25539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:12.876134 25539 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:42681
I20260808 10:44:12.876422 25610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:42681 every 8 connection(s)
I20260808 10:44:12.877110 25539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:12.879937 25611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:12.883627 25611 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:12.884229 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25539
I20260808 10:44:12.884372 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:12.884619 1801 external_mini_cluster.cc:1428] Setting key c4ede123570bb5af23d2d26529c1509d
WARNING: no policy specified for kudu/127.1.194.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.125@KRBTEST.COM" created.
Aug 08 10:44:12 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:44:12.904317 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:12.885948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42681 (local address 127.0.0.1:42274)
0808 10:44:12.886401 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:12.886452 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:12.887274 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:12.888123 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:12.888131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:12.888609 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:12.888982 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:12.889004 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:12.890369 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:12.890375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:12.891701 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:12.891712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:12.892439 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:12.892444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:12.892589 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:12.893273 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:12.893300 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:12.898588 (+ 5288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:12.900896 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:12.900914 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:12.900928 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:12.901283 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:12.901750 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:12.901753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:12.901755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:12.901876 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:12.902527 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:12.902534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:12.902844 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 10:44:12.903095 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 10:44:12.903412 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":127,"threads_started":1}
I20260808 10:44:12.904321 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:12.885948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.1.194.126:42681)
0808 10:44:12.886487 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:12.886500 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:12.886551 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:44:12.887598 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:12.887606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:12.887861 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:12.888046 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:12.889177 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:12.890217 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:12.891854 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:12.892260 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:12.892394 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:12.898796 (+ 6402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:12.898807 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:12.898826 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:12.898862 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:12.900748 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:12.901400 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:12.901404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:12.901405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:12.901639 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:12.901983 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:12.901986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:12.901988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:12.902400 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:12.902590 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:12.903265 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 10:44:12.903561 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":413,"thread_start_us":164,"threads_started":1}
I20260808 10:44:12.907059 25611 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:12.907930 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:12.907598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36128)
0808 10:44:12.907752 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:12.907840 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 10:44:12.908162 25558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
I20260808 10:44:12.909017 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:12.908821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36140)
0808 10:44:12.908923 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:12.908967 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:44:12.909175 25611 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:44929: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab.
Entry for principal kudu/127.1.194.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab.
I20260808 10:44:12.917218 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:44929
--webserver_interface=127.1.194.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab
--principal=kudu/127.1.194.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42681,127.1.194.125:44929,127.1.194.124:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:44:12.958442 25611 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } attempt: 1
I20260808 10:44:12.959435 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:12.959020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36148)
0808 10:44:12.959223 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:12.959319 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 10:44:12.960332 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:12.960038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36164)
0808 10:44:12.960178 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:12.960245 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:44:12.960496 25611 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:44929: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
I20260808 10:44:13.028677 25611 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } attempt: 2
I20260808 10:44:13.029680 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.029197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36174)
0808 10:44:13.029378 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.029455 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 10:44:13.030072 25618 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:44:13.030405 25618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:13.030481 25618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:13.030544 25618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:44:13.030555 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.030211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36190)
0808 10:44:13.030318 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.030399 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 10:44:13.030743 25611 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:44929: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44929: connect: Connection refused (error 111)
W20260808 10:44:13.034685 25618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:13.034765 25618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:13.034811 25618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:13.034847 25618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:13.034879 25618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:13.040143 25618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42681,127.1.194.125:44929,127.1.194.124:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab
--principal=kudu/127.1.194.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:44929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.194.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:13.041244 25618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:13.042327 25618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:13.050952 25626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:13.051003 25624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:13.051051 25623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:13.051313 25618 server_base.cc:1034] running on GCE node
I20260808 10:44:13.051895 25618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:13.053045 25618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:13.054252 25618 hybrid_clock.cc:648] HybridClock initialized: now 1786185853054228 us; error 26 us; skew 500 ppm
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:13.057353 25618 init.cc:377] Logged in from keytab as kudu/127.1.194.125@KRBTEST.COM (short username kudu)
I20260808 10:44:13.058327 25618 webserver.cc:460] Webserver started at http://127.1.194.125:35339/ using document root <none> and password file <none>
I20260808 10:44:13.058885 25618 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:13.058949 25618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:13.059161 25618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:13.061177 25618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data/instance:
uuid: "aff8d82685984d8d854ba18baef36a15"
format_stamp: "Formatted at 2026-08-08 10:44:13 on dist-test-slave-ss4n"
server_key: "26e1bd741ca7306f7718b9ae4d36fc89"
server_key_iv: "2b8c99acc9cdf2f8c50449aa3aa91c1a"
server_key_version: "encryptionkey@0"
I20260808 10:44:13.061777 25618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal/instance:
uuid: "aff8d82685984d8d854ba18baef36a15"
format_stamp: "Formatted at 2026-08-08 10:44:13 on dist-test-slave-ss4n"
server_key: "26e1bd741ca7306f7718b9ae4d36fc89"
server_key_iv: "2b8c99acc9cdf2f8c50449aa3aa91c1a"
server_key_version: "encryptionkey@0"
I20260808 10:44:13.065452 25618 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:44:13.068090 25633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:13.069391 25618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:44:13.069574 25618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal
uuid: "aff8d82685984d8d854ba18baef36a15"
format_stamp: "Formatted at 2026-08-08 10:44:13 on dist-test-slave-ss4n"
server_key: "26e1bd741ca7306f7718b9ae4d36fc89"
server_key_iv: "2b8c99acc9cdf2f8c50449aa3aa91c1a"
server_key_version: "encryptionkey@0"
I20260808 10:44:13.069692 25618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:13.081632 25618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:13.084872 25618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:13.085062 25618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:13.092291 25618 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.125:44929
I20260808 10:44:13.092620 25688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.125:44929 every 8 connection(s)
I20260808 10:44:13.093263 25618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
I20260808 10:44:13.094436 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25618
I20260808 10:44:13.094661 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-1/wal/instance
I20260808 10:44:13.094945 1801 external_mini_cluster.cc:1428] Setting key 0ccb975e368d1a455d329384671cd6a3
I20260808 10:44:13.096181 25689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:13.101260 25689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.194.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.124@KRBTEST.COM" created.
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:44:13.116632 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.103260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.1.194.126:42681)
0808 10:44:13.103404 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.103408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.104564 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 10:44:13.105564 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.105568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.105646 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.105739 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.106940 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.107580 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.108714 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.108953 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.109068 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.111942 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.111945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.111947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.111972 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.114158 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.114811 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.114813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.114815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.114893 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.115311 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.115314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.115315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.115489 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.115562 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.116360 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.116497 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:44:13.117569 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.103782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42681 (local address 127.0.0.1:42282)
0808 10:44:13.104397 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.104456 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.105290 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.105906 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.105913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.106239 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.106799 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.106814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.107714 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.107717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.108585 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.108594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.109109 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.109113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.109292 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.109949 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.109965 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.111755 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.114286 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.114294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.114313 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.114697 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.114993 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.114997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.114999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.115207 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.115596 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.115603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.115976 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.116257 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.116632 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":179,"threads_started":1}
I20260808 10:44:13.119282 25689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:13.121902 25611 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } attempt: 3
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab.
Entry for principal kudu/127.1.194.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab.
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
I20260808 10:44:13.128520 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:39071
--webserver_interface=127.1.194.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab
--principal=kudu/127.1.194.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:42681,127.1.194.125:44929,127.1.194.124:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:44:13.132169 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.119859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36202)
0808 10:44:13.120073 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.120086 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.120172 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.120747 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.120749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.120886 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.120985 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.120990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.122016 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.122021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.122870 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.122879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.123554 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.123557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.123650 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.124224 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.124235 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.125569 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.130173 (+ 4604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.130180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.130185 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.130508 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.130850 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.130856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.130859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.130963 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.131574 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.131579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.131690 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.131818 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.131962 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260808 10:44:13.132571 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.120010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36202 (local address 127.1.194.125:44929)
0808 10:44:13.120339 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.120349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.120369 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:44:13.120407 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.120411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.120612 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.120755 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.121099 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.121889 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.122985 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.123445 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.123561 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.125704 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.125714 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.125728 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.125752 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.130048 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.130644 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.130650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.130652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.130747 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.131083 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.131088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.131090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.131441 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.131604 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.131932 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.132095 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":160,"threads_started":1}
I20260808 10:44:13.133625 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.122379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36208)
0808 10:44:13.122510 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.122526 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.122621 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.123127 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.123130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.123349 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.123447 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.123454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.124482 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.124497 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.125192 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.125202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.125872 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.125875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.125958 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.126701 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.126718 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.128273 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.131641 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.131645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.131647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.131946 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.132427 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.132431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.132433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.132531 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.133030 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.133034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.133143 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.133262 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.133413 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 10:44:13.133651 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.122473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.1.194.125:44929)
0808 10:44:13.122911 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.122918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.122929 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:44:13.122958 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.122960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.123001 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.123108 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.123581 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.124337 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.125345 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.125759 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.125860 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.128434 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.128438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.128441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.128466 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.131486 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.132086 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.132089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.132091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.132170 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.132680 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.132685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.132688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.132898 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.133040 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.133317 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.133430 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":141,"threads_started":1}
I20260808 10:44:13.134660 25689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:13.135128 25611 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:13.135437 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.135118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51136)
0808 10:44:13.135264 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.135357 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:44:13.135790 25637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
I20260808 10:44:13.135886 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.135591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51148)
0808 10:44:13.135737 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.135800 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 10:44:13.136739 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.136460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51154)
0808 10:44:13.136603 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.136664 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 10:44:13.137131 25689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:39071: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
I20260808 10:44:13.137241 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.136922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51156)
0808 10:44:13.137104 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.137166 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260808 10:44:13.137420 25611 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:39071: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
I20260808 10:44:13.168623 25611 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } attempt: 1
I20260808 10:44:13.169622 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.169216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51160)
0808 10:44:13.169374 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.169441 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 10:44:13.170408 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.170210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51174)
0808 10:44:13.170309 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.170356 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:44:13.170636 25611 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:39071: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
I20260808 10:44:13.186435 25689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } attempt: 1
I20260808 10:44:13.187520 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.187184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51190)
0808 10:44:13.187350 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.187422 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 10:44:13.188306 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.188077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51206)
0808 10:44:13.188193 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.188246 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:44:13.188465 25689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:39071: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
W20260808 10:44:13.242231 25696 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:44:13.242506 25696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:13.242563 25696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:13.242652 25696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:44:13.245807 25611 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } attempt: 2
I20260808 10:44:13.246727 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.246357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51210)
0808 10:44:13.246524 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.246600 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 10:44:13.247023 25696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:13.247080 25696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:13.247107 25696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:13.247130 25696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:13.247150 25696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:13.247605 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.247358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51224)
0808 10:44:13.247491 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.247548 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 10:44:13.247756 25611 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:39071: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
I20260808 10:44:13.252552 25696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:42681,127.1.194.125:44929,127.1.194.124:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab
--principal=kudu/127.1.194.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.194.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:13.253671 25696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:13.254695 25696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:44:13.256716 25689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } attempt: 2
I20260808 10:44:13.257566 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.257218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51238)
0808 10:44:13.257381 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.257448 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:44:13.258347 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.258110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51254)
0808 10:44:13.258224 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.258279 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 10:44:13.258502 25689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:39071: Network error: Client connection negotiation failed: client connection to 127.1.194.124:39071: connect: Connection refused (error 111)
W20260808 10:44:13.263628 25703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:13.263675 25702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:13.263638 25705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:13.263926 25696 server_base.cc:1034] running on GCE node
I20260808 10:44:13.264523 25696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:13.265681 25696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:13.266955 25696 hybrid_clock.cc:648] HybridClock initialized: now 1786185853266937 us; error 29 us; skew 500 ppm
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:13.270072 25696 init.cc:377] Logged in from keytab as kudu/127.1.194.124@KRBTEST.COM (short username kudu)
I20260808 10:44:13.270974 25696 webserver.cc:460] Webserver started at http://127.1.194.124:42739/ using document root <none> and password file <none>
I20260808 10:44:13.271509 25696 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:13.271574 25696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:13.271739 25696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:13.274001 25696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data/instance:
uuid: "8e9958efa2ca40b9b5f89a9d10525285"
format_stamp: "Formatted at 2026-08-08 10:44:13 on dist-test-slave-ss4n"
server_key: "899f939af0ae53af3dd872506fc620f0"
server_key_iv: "f0853369ce3dc43239bd80d1fc1bcdb4"
server_key_version: "encryptionkey@0"
I20260808 10:44:13.274631 25696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal/instance:
uuid: "8e9958efa2ca40b9b5f89a9d10525285"
format_stamp: "Formatted at 2026-08-08 10:44:13 on dist-test-slave-ss4n"
server_key: "899f939af0ae53af3dd872506fc620f0"
server_key_iv: "f0853369ce3dc43239bd80d1fc1bcdb4"
server_key_version: "encryptionkey@0"
I20260808 10:44:13.278558 25696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:44:13.280900 25713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:13.282279 25696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:13.282377 25696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal
uuid: "8e9958efa2ca40b9b5f89a9d10525285"
format_stamp: "Formatted at 2026-08-08 10:44:13 on dist-test-slave-ss4n"
server_key: "899f939af0ae53af3dd872506fc620f0"
server_key_iv: "f0853369ce3dc43239bd80d1fc1bcdb4"
server_key_version: "encryptionkey@0"
I20260808 10:44:13.282460 25696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:13.297691 25696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:13.300920 25696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:13.301077 25696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:13.308228 25696 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.124:39071
I20260808 10:44:13.308591 25768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.124:39071 every 8 connection(s)
I20260808 10:44:13.309211 25696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
I20260808 10:44:13.312282 25769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:13.316495 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25696
I20260808 10:44:13.316267 25769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:13.316653 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185564023815-1801-0/minicluster-data/master-2/wal/instance
I20260808 10:44:13.316958 1801 external_mini_cluster.cc:1428] Setting key a3b5b9b0da84798517f2587a45ec0ada
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:13.330751 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.318343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.1.194.126:42681)
0808 10:44:13.318474 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.318478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.319168 (+ 690us) server_negotiation.cc:407] Connection header received
0808 10:44:13.320340 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.320343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.320394 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.320482 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.321539 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.322134 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.323305 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.323663 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.323751 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.326733 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.326739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.326743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.326780 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.328355 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.329019 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.329024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.329026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.329112 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.329525 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.329531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.329536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.329749 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.329892 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.330421 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.330562 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:44:13.331568 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.318480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.1.194.124:39071)
0808 10:44:13.319354 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.319366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.319402 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:44:13.320104 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.320128 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.320382 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.320586 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.321380 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.322520 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.323599 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.324107 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.324225 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.327443 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.327448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.327458 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.327478 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.329121 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.329594 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.329606 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.329608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.329687 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.329979 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.329983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.329984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.330231 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.330436 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.330676 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.330818 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"spinlock_wait_cycles":72704,"thread_start_us":176,"threads_started":1}
I20260808 10:44:13.331566 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.318512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:42681 (local address 127.0.0.1:42284)
0808 10:44:13.319019 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.319060 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.320104 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.320556 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.320562 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.321044 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.321325 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.321348 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.322265 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.322269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.323177 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.323187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.323806 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.323810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.324028 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.324766 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.324783 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.326573 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.328590 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.328599 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.328610 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.328892 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.329213 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.329217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.329220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.329368 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.329888 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.329895 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.330163 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.330363 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.330683 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":176,"threads_started":1}
I20260808 10:44:13.333066 25769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:44:13.342303 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.333628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44929 (local address 127.0.0.1:36214)
0808 10:44:13.333843 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.333859 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.333950 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.334342 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.334345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.334550 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.334678 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.334685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.335486 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.335493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.336067 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.336074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.336597 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.336601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.336710 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.337234 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.337254 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.338722 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.340732 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.340735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.340736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.340928 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.341272 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.341275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.341276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.341375 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.341861 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.341864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.341963 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.342073 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.342190 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 10:44:13.342463 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.333732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36214 (local address 127.1.194.125:44929)
0808 10:44:13.333897 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.333902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.333919 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:44:13.334076 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.334080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.334136 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.334216 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.334799 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.335364 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.336194 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.336477 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.336599 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.338864 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.338867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.338869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.338892 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.340609 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.341033 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.341038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.341042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.341141 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.341471 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.341473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.341474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.341757 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.341860 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.342145 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.342279 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:44:13.343216 25769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:13.343444 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.334146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.1.194.126:42681)
0808 10:44:13.334265 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.334270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.334282 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:44:13.334405 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.334408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.334453 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.334538 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.335125 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.335597 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.336367 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.336630 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.336747 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.339639 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.339646 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.339652 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.339682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.341168 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.341668 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.341671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.341673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.341750 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.342065 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.342069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.342072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.342304 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.342530 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.343195 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.343311 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:44:13.346920 25611 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } attempt: 3
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:13.349669 25689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } attempt: 3
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185852, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:13.354108 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.344052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.1.194.124:39071)
0808 10:44:13.344165 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.344172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.344183 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:44:13.344213 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.344216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.344251 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.344328 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.344812 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.345347 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.346195 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.346600 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.346687 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.349195 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.349198 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.349200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.349221 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.352052 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.352788 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.352795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.352797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.352881 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.353194 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.353198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.353200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.353386 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.353457 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.353791 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.353898 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:44:13.354616 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.343800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51268)
0808 10:44:13.343914 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.343925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.344012 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.344347 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.344349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.344571 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.344674 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.344681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.345456 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.345461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.346084 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.346093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.346716 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.346719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.346798 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.347326 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.347339 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.348889 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.352182 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.352189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.352192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.352679 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.352990 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.352993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.352995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.353092 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.353549 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.353552 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.353631 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.354334 (+ 703us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.354464 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
Aug 08 10:44:13 dist-test-slave-ss4n krb5kdc[25525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:13.357586 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.347373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51270)
0808 10:44:13.347510 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.347527 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.347618 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.348153 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.348157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.348332 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.348449 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.348460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.349620 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.349658 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.350495 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.350510 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.351161 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.351166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.351254 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.351837 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.351848 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.353304 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.355518 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.355523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.355526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.355802 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.356411 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.356415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.356417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.356506 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.356971 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.356974 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.357092 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.357217 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.357367 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:44:13.357607 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.347440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.1.194.124:39071)
0808 10:44:13.347877 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.347887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.347900 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:44:13.347938 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.347944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.348000 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.348101 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.348553 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.349431 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.350664 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.351036 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.351449 (+ 413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.353461 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.353467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.353472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.353528 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.355396 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.356193 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.356199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.356203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.356292 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.356615 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.356618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.356620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.356846 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.356941 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.357293 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.357409 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":143,"threads_started":1}
I20260808 10:44:13.358516 25769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: Bootstrap starting.
I20260808 10:44:13.361002 25611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8: Bootstrap starting.
I20260808 10:44:13.361127 25769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:13.361523 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.350101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:39071 (local address 127.0.0.1:51280)
0808 10:44:13.350382 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:13.350396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:13.350498 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:13.352400 (+ 1902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:13.352404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:13.352577 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:13.352682 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.352688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.354041 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.354045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.354888 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:13.354898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.355661 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.355669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:13.355765 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.356201 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:13.356212 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:13.357676 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:13.359828 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.359832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.359834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.360076 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.360400 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:13.360403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:13.360405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:13.360497 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:13.360947 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:13.360953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:13.361069 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 10:44:13.361192 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:44:13.361326 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260808 10:44:13.361598 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.350666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.1.194.124:39071)
0808 10:44:13.351205 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.351215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.351231 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:13.352210 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.352213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.352262 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.352359 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.352996 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.353908 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.355148 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.355504 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.355968 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.357799 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.357805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.357810 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.357840 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.359701 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.360190 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.360193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.360195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.360287 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.360600 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.360603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.360605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.360820 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.360938 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.361265 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.361386 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":126,"threads_started":1}
I20260808 10:44:13.362682 25769 log.cc:824] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:13.363651 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.344674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.1.194.125:44929)
0808 10:44:13.344780 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.344786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.344799 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:44:13.344932 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.344935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.344971 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.345088 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.345761 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.346351 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.347695 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.347990 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.348048 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.351147 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.351152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.351154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.351178 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.353625 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.354460 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.354464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.354467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.354569 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.355170 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.355175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.355177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.355394 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.355603 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.363418 (+ 7815us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.363548 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:44:13.363831 25611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:13.364579 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: Bootstrap starting.
I20260808 10:44:13.364727 25611 log.cc:824] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:13.365036 25769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: No bootstrap required, opened a new log
I20260808 10:44:13.366880 25611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8: No bootstrap required, opened a new log
I20260808 10:44:13.367224 25689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:13.368085 25689 log.cc:824] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:13.368872 25769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.369104 25769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9958efa2ca40b9b5f89a9d10525285, State: Initialized, Role: FOLLOWER
I20260808 10:44:13.369418 25769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.370247 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: No bootstrap required, opened a new log
I20260808 10:44:13.370649 25769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:13.371168 25611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.371394 25611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841a1d698e694bcc8c2cbfa538ed0df8, State: Initialized, Role: FOLLOWER
I20260808 10:44:13.371721 25611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.372236 25785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.372474 25785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:13.373817 25611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:13.375058 25689 raft_consensus.cc:348] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.375288 25689 raft_consensus.cc:729] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff8d82685984d8d854ba18baef36a15, State: Initialized, Role: FOLLOWER
I20260808 10:44:13.375607 25689 consensus_queue.cc:260] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.376566 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:13.376888 25689 sys_catalog.cc:564] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:44:13.376950 25791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:13.377153 25791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:13.374502 25696 master_runner.cc:186] Error getting master registration for 127.1.194.126:42681: OK, instance_id { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" instance_seqno: 1786185852874174 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:44:13.378032 25583 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "841a1d698e694bcc8c2cbfa538ed0df8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42300
I20260808 10:44:13.378286 25696 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:44:13.378345 25583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:44:13.378472 25583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:13.379849 25583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.380810 25583 leader_election.cc:288] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [CANDIDATE]: Term 1 election: Requested vote from peers aff8d82685984d8d854ba18baef36a15 (127.1.194.125:44929), 8e9958efa2ca40b9b5f89a9d10525285 (127.1.194.124:39071)
I20260808 10:44:13.381333 25779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.381508 25779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:13.382149 25742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8e9958efa2ca40b9b5f89a9d10525285"
I20260808 10:44:13.382393 25742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [term 0 FOLLOWER]: Advancing to term 1
W20260808 10:44:13.383587 25817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:13.383674 25817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:13.384143 25742 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 841a1d698e694bcc8c2cbfa538ed0df8 in term 1.
I20260808 10:44:13.384583 25558 leader_election.cc:302] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841a1d698e694bcc8c2cbfa538ed0df8, 8e9958efa2ca40b9b5f89a9d10525285; no voters:
I20260808 10:44:13.381810 25660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "aff8d82685984d8d854ba18baef36a15"
I20260808 10:44:13.384855 25785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:13.384940 25660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:13.385058 25785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [term 1 LEADER]: Becoming Leader. State: Replica: 841a1d698e694bcc8c2cbfa538ed0df8, State: Running, Role: LEADER
I20260808 10:44:13.385367 25785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } }
I20260808 10:44:13.386302 25660 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 841a1d698e694bcc8c2cbfa538ed0df8 in term 1.
I20260808 10:44:13.386487 25793 sys_catalog.cc:455] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.386622 25793 sys_catalog.cc:458] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:13.387159 25785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 841a1d698e694bcc8c2cbfa538ed0df8. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.387264 25785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:13.394833 25822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:13.395845 25822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:13.401782 25740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [term 1 FOLLOWER]: Refusing update from remote peer 841a1d698e694bcc8c2cbfa538ed0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:13.402438 25785 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:13.404767 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.390517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.1.194.125:44929)
0808 10:44:13.390662 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.390667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.394481 (+ 3814us) server_negotiation.cc:407] Connection header received
0808 10:44:13.394591 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.394598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.394653 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.394742 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.395868 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.396764 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.398073 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.398432 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.398518 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.400712 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.400716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.400719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.400748 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.402751 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.403297 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.403301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.403303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.403391 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.403729 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.403733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.403735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.403954 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.404077 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.404461 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.404593 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:44:13.406365 25779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 841a1d698e694bcc8c2cbfa538ed0df8. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.406484 25779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:44:13.409667 25810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:13.409751 25810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:13.410714 25660 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [term 1 FOLLOWER]: Refusing update from remote peer 841a1d698e694bcc8c2cbfa538ed0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:13.411319 25785 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:13.415366 25793 sys_catalog.cc:455] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 841a1d698e694bcc8c2cbfa538ed0df8. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.415516 25793 sys_catalog.cc:458] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:13.419224 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.419384 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:13.420543 25779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.420665 25779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:13.421208 25830 mvcc.cc:205] Tried to move back new op lower bound from 7316217255526391808 to 7316217255469957120. Current Snapshot: MvccSnapshot[applied={T|T < 7316217255526391808}]
I20260808 10:44:13.421612 25831 mvcc.cc:205] Tried to move back new op lower bound from 7316217255526391808 to 7316217255469957120. Current Snapshot: MvccSnapshot[applied={T|T < 7316217255526391808}]
I20260808 10:44:13.421744 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.394135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.1.194.126:42681)
0808 10:44:13.394246 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.394250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.399491 (+ 5241us) server_negotiation.cc:407] Connection header received
0808 10:44:13.399698 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.399702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.399756 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.399816 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.409601 (+ 9785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.410418 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.413045 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.413416 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.413539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.416817 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.416825 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.416832 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.416869 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.418762 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.419319 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.419329 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.419333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.419437 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.419831 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.419834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.419836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.420114 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.420258 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.420958 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.421111 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:44:13.423864 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:13.390910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.1.194.124:39071)
0808 10:44:13.391017 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.391021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.397829 (+ 6808us) server_negotiation.cc:407] Connection header received
0808 10:44:13.398016 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.398021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.398079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.398132 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.400547 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.401468 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.411674 (+ 10206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:13.412089 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:13.412184 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:13.416392 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:13.416396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:13.416400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:13.416433 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:13.420263 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.422248 (+ 1985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.422252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.422254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.422357 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:13.422896 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:13.422900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:13.422902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:13.423123 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:13.423264 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:13.423593 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 10:44:13.423719 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:44:13.426545 25785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.426659 25785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:13.427778 25793 sys_catalog.cc:455] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841a1d698e694bcc8c2cbfa538ed0df8" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 42681 } } peers { permanent_uuid: "aff8d82685984d8d854ba18baef36a15" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44929 } } peers { permanent_uuid: "8e9958efa2ca40b9b5f89a9d10525285" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 39071 } } }
I20260808 10:44:13.427908 25793 sys_catalog.cc:458] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:13.432634 25822 catalog_manager.cc:1347] Generated new cluster ID: c91ed75db04c43b0bda27bc240cf1748
I20260808 10:44:13.432698 25822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:13.465305 25822 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:13.467382 25822 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:13.478797 25822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 841a1d698e694bcc8c2cbfa538ed0df8: Generated new TSK 0
I20260808 10:44:13.479954 25822 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:44:13.522837 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.519287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.1.194.125:44929)
0808 10:44:13.519423 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.519428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.521527 (+ 2099us) server_negotiation.cc:407] Connection header received
0808 10:44:13.521819 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.521823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.521878 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.521937 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.522687 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36246: BlockingRecv error: recv got EOF from 127.0.0.1:36246 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 10:44:13.524266 25775 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.520101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.1.194.124:39071)
0808 10:44:13.520217 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.520222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.523293 (+ 3071us) server_negotiation.cc:407] Connection header received
0808 10:44:13.523476 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.523479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.523534 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.523589 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.524152 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51292: BlockingRecv error: recv got EOF from 127.0.0.1:51292 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:44:13.525843 25614 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.520839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.1.194.126:42681)
0808 10:44:13.520964 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.520968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.524800 (+ 3832us) server_negotiation.cc:407] Connection header received
0808 10:44:13.525066 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.525070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.525123 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.525179 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.525720 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42318: BlockingRecv error: recv got EOF from 127.0.0.1:42318 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 10:44:13.530663 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.529041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.1.194.125:44929)
0808 10:44:13.529154 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.529160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.529673 (+ 513us) server_negotiation.cc:407] Connection header received
0808 10:44:13.529902 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.529906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.529998 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.530056 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.530558 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36250: BlockingRecv error: recv got EOF from 127.0.0.1:36250 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 10:44:13.535715 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.534149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.1.194.125:44929)
0808 10:44:13.534254 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.534259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.534803 (+ 544us) server_negotiation.cc:407] Connection header received
0808 10:44:13.535006 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.535010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.535061 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.535114 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.535615 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36254: BlockingRecv error: recv got EOF from 127.0.0.1:36254 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:44:13.541332 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.539724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.1.194.125:44929)
0808 10:44:13.539828 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.539833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.540380 (+ 547us) server_negotiation.cc:407] Connection header received
0808 10:44:13.540573 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.540576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.540626 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.540679 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.541230 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36270: BlockingRecv error: recv got EOF from 127.0.0.1:36270 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:44:13.549234 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.547440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36280 (local address 127.1.194.125:44929)
0808 10:44:13.547590 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.547594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.548211 (+ 617us) server_negotiation.cc:407] Connection header received
0808 10:44:13.548424 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.548429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.548485 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.548547 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.549120 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36280: BlockingRecv error: recv got EOF from 127.0.0.1:36280 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 10:44:13.561779 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.559766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36294 (local address 127.1.194.125:44929)
0808 10:44:13.559894 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.559900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.560599 (+ 699us) server_negotiation.cc:407] Connection header received
0808 10:44:13.560821 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.560826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.560889 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.560953 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.561644 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36294: BlockingRecv error: recv got EOF from 127.0.0.1:36294 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 10:44:13.582777 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.580641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.1.194.125:44929)
0808 10:44:13.580783 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.580788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.581551 (+ 763us) server_negotiation.cc:407] Connection header received
0808 10:44:13.581810 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.581815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.581884 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.581951 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.582644 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36298: BlockingRecv error: recv got EOF from 127.0.0.1:36298 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 10:44:13.618948 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.617651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.1.194.125:44929)
0808 10:44:13.617798 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.617803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.618014 (+ 211us) server_negotiation.cc:407] Connection header received
0808 10:44:13.618180 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.618184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.618245 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.618337 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.618831 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36304: BlockingRecv error: recv got EOF from 127.0.0.1:36304 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 10:44:13.687011 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.685640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.1.194.125:44929)
0808 10:44:13.685787 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.685792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.685982 (+ 190us) server_negotiation.cc:407] Connection header received
0808 10:44:13.686161 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.686165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.686225 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.686319 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.686852 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36314: BlockingRecv error: recv got EOF from 127.0.0.1:36314 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 10:44:13.818686 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:13.817379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.1.194.125:44929)
0808 10:44:13.817606 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:13.817610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:13.817739 (+ 129us) server_negotiation.cc:407] Connection header received
0808 10:44:13.817871 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:13.817874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:13.817921 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:13.818038 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:13.818558 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36322: BlockingRecv error: recv got EOF from 127.0.0.1:36322 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260808 10:44:14.078629 25694 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:14.077156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.1.194.125:44929)
0808 10:44:14.077379 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:14.077384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:14.077610 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:44:14.077828 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:14.077832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:14.077879 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:14.077998 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:14.078496 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36338: BlockingRecv error: recv got EOF from 127.0.0.1:36338 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:44:14.378715 25791 catalog_manager.cc:1259] Loaded cluster ID: c91ed75db04c43b0bda27bc240cf1748
I20260808 10:44:14.378813 25791 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: loading cluster ID for follower catalog manager: success
I20260808 10:44:14.380702 25791 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: acquiring CA information for follower catalog manager: success
I20260808 10:44:14.381944 25791 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e9958efa2ca40b9b5f89a9d10525285: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:44:14.411130 25810 catalog_manager.cc:1259] Loaded cluster ID: c91ed75db04c43b0bda27bc240cf1748
I20260808 10:44:14.411222 25810 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: loading cluster ID for follower catalog manager: success
I20260808 10:44:14.412894 25810 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: acquiring CA information for follower catalog manager: success
I20260808 10:44:14.414140 25810 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aff8d82685984d8d854ba18baef36a15: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:44:14.597383 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:14.593022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.1.194.125:44929)
0808 10:44:14.593321 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:14.593325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:14.593592 (+ 267us) server_negotiation.cc:407] Connection header received
0808 10:44:14.593657 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:14.593660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:14.593790 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:14.593910 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:14.594235 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.594785 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.595807 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.596024 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.596103 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:14.596407 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:14.596839 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:14.596931 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:14.597121 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:44:14.597172 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":86,"threads_started":1}
I20260808 10:44:14.604756 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:14.600818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51298 (local address 127.1.194.124:39071)
0808 10:44:14.601079 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:14.601084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:14.601100 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:14.601196 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:14.601204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:14.601311 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:14.601400 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:14.601769 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.602312 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.603147 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.603373 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.603464 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:14.603806 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:14.604238 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:14.604370 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:14.604441 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:44:14.604491 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260808 10:44:14.612397 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:14.608413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.1.194.124:39071)
0808 10:44:14.608558 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:14.608562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:14.608872 (+ 310us) server_negotiation.cc:407] Connection header received
0808 10:44:14.608916 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:14.608918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:14.608977 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:14.609053 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:14.609399 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.610134 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.611247 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.611578 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.611647 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:14.611976 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:14.612050 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:14.612120 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:14.612248 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 10:44:14.612297 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:44:14.612383 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:14.607872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.1.194.125:44929)
0808 10:44:14.608022 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:14.608027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:14.608267 (+ 240us) server_negotiation.cc:407] Connection header received
0808 10:44:14.608657 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:14.608661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:14.608712 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:14.608789 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:14.609132 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.609818 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.611078 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.611398 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.611468 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:14.611865 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:14.611950 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:14.611999 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:14.612194 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:44:14.612257 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:44:14.612507 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:14.607853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.1.194.126:42681)
0808 10:44:14.608131 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:14.608137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:14.608151 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:44:14.608359 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:14.608364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:14.608539 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:14.608659 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:14.609052 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.609821 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.610833 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:14.611078 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:14.611196 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:14.611503 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:14.611762 (+ 259us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:14.611869 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:14.612136 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 10:44:14.612206 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260808 10:44:14.615972 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25539
I20260808 10:44:14.624469 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25618
I20260808 10:44:14.631070 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25696
2026-08-08T10:44:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3175 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:14 dist-test-slave-ss4n krb5kdc[25925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:14 dist-test-slave-ss4n krb5kdc[25925](info): set up 2 sockets
Aug 08 10:44:14 dist-test-slave-ss4n krb5kdc[25925](info): commencing operation
krb5kdc: starting...
W20260808 10:44:15.728613 1801 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/test-user.keytab.
Aug 08 10:44:15 dist-test-slave-ss4n krb5kdc[25925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.194.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
Entry for principal kudu/127.1.194.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab.
I20260808 10:44:15.866057 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46127
--webserver_interface=127.1.194.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:46127,127.1.194.125:44231,127.1.194.124:37701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:15.983634 25939 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:44:15.983999 25939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:15.984074 25939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:15.984169 25939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:15.988296 25939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:15.988389 25939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:15.988427 25939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:15.988474 25939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:15.988505 25939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:15.993901 25939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:46127,127.1.194.125:44231,127.1.194.124:37701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.126.keytab
--principal=kudu/127.1.194.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.126:46127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.194.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:15.995028 25939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:15.996069 25939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:16.004972 25947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:16.004983 25944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:16.004988 25945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:16.005112 25939 server_base.cc:1034] running on GCE node
I20260808 10:44:16.005867 25939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:16.007133 25939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:16.008343 25939 hybrid_clock.cc:648] HybridClock initialized: now 1786185856008313 us; error 50 us; skew 500 ppm
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:16.011647 25939 init.cc:377] Logged in from keytab as kudu/127.1.194.126@KRBTEST.COM (short username kudu)
I20260808 10:44:16.012530 25939 webserver.cc:460] Webserver started at http://127.1.194.126:38177/ using document root <none> and password file <none>
I20260808 10:44:16.013073 25939 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:16.013134 25939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:16.013296 25939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:16.015365 25939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data/instance:
uuid: "b34cb677fb834e6e8b781dba0847d1e0"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "699cd5fae1059939b95ec8a89b9deff7"
server_key_iv: "4e9de02d63a7818de633d6bc084d5cac"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.015903 25939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal/instance:
uuid: "b34cb677fb834e6e8b781dba0847d1e0"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "699cd5fae1059939b95ec8a89b9deff7"
server_key_iv: "4e9de02d63a7818de633d6bc084d5cac"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.019937 25939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:16.022647 25956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:16.023949 25939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:16.024089 25939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal
uuid: "b34cb677fb834e6e8b781dba0847d1e0"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "699cd5fae1059939b95ec8a89b9deff7"
server_key_iv: "4e9de02d63a7818de633d6bc084d5cac"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.024206 25939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:16.062314 25939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:16.066150 25939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:16.066335 25939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:16.073803 25939 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.126:46127
I20260808 10:44:16.074129 26010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.126:46127 every 8 connection(s)
I20260808 10:44:16.074883 25939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:16.077895 26011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:16.081609 26011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.083930 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 25939
I20260808 10:44:16.084064 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-0/wal/instance
I20260808 10:44:16.084376 1801 external_mini_cluster.cc:1428] Setting key 43b6ffd0cb2fb3139374e282b1b7c5dd
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.125@KRBTEST.COM" created.
I20260808 10:44:16.099045 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.084756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.1.194.126:46127)
0808 10:44:16.085100 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.085114 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.085166 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:44:16.085841 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.085850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.086117 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.086331 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.087434 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.088420 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.089458 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.090249 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.090464 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.093206 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.093213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.093227 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.093259 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.095545 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.096051 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.096057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.096058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.096176 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.096649 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.096653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.096655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.097053 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.097219 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.097645 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.097918 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260808 10:44:16.099046 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.083871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46127 (local address 127.0.0.1:59130)
0808 10:44:16.084401 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.084453 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.085357 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.086347 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.086352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.086856 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.087255 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.087275 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.088549 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.088552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.089319 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.089329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.090409 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.090412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.090493 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.091188 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.091203 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.093034 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.095659 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.095668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.095676 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.095913 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.096308 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.096311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.096313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.096439 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.097188 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.097195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.097344 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.097566 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.097984 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":120,"thread_start_us":172,"threads_started":1}
I20260808 10:44:16.101518 26011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.102445 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.102050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37302)
0808 10:44:16.102235 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.102344 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 10:44:16.102735 25958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
I20260808 10:44:16.103767 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.103473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37308)
0808 10:44:16.103615 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.103686 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:44:16.103930 26011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:44231: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab.
Entry for principal kudu/127.1.194.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab.
I20260808 10:44:16.118153 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:44231
--webserver_interface=127.1.194.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab
--principal=kudu/127.1.194.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:46127,127.1.194.125:44231,127.1.194.124:37701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:44:16.153213 26011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } attempt: 1
I20260808 10:44:16.154237 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.153924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37314)
0808 10:44:16.154077 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.154157 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 10:44:16.155090 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.154843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37328)
0808 10:44:16.154966 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.155033 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 10:44:16.155221 26011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:44231: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
I20260808 10:44:16.223415 26011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } attempt: 2
I20260808 10:44:16.224326 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.224027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37340)
0808 10:44:16.224177 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.224249 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 10:44:16.225098 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.224863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37350)
0808 10:44:16.225001 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.225045 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:44:16.225222 26011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.125:44231: Network error: Client connection negotiation failed: client connection to 127.1.194.125:44231: connect: Connection refused (error 111)
W20260808 10:44:16.230964 26019 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:44:16.231314 26019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:16.231379 26019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:16.231462 26019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:16.235687 26019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:16.235771 26019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:16.235819 26019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:16.235875 26019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:16.235913 26019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:16.241356 26019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:46127,127.1.194.125:44231,127.1.194.124:37701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.125.keytab
--principal=kudu/127.1.194.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.125:44231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.194.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:16.242540 26019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:16.243574 26019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:16.252317 26028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:16.252305 26024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:16.252365 26025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:16.252661 26019 server_base.cc:1034] running on GCE node
I20260808 10:44:16.253309 26019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:16.254638 26019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:16.255865 26019 hybrid_clock.cc:648] HybridClock initialized: now 1786185856255847 us; error 33 us; skew 500 ppm
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:16.259045 26019 init.cc:377] Logged in from keytab as kudu/127.1.194.125@KRBTEST.COM (short username kudu)
I20260808 10:44:16.260090 26019 webserver.cc:460] Webserver started at http://127.1.194.125:33081/ using document root <none> and password file <none>
I20260808 10:44:16.260713 26019 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:16.260828 26019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:16.261093 26019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:16.263402 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data/instance:
uuid: "d91838935d4f4a3388085d09aeb887ab"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "7bfa982e0876da0f5a7cf51f5c8e4f8a"
server_key_iv: "110b62098e788f3f8b3b4d5c3022aad8"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.264045 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal/instance:
uuid: "d91838935d4f4a3388085d09aeb887ab"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "7bfa982e0876da0f5a7cf51f5c8e4f8a"
server_key_iv: "110b62098e788f3f8b3b4d5c3022aad8"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.268232 26019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 10:44:16.270922 26035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:16.272305 26019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:44:16.272480 26019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal
uuid: "d91838935d4f4a3388085d09aeb887ab"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "7bfa982e0876da0f5a7cf51f5c8e4f8a"
server_key_iv: "110b62098e788f3f8b3b4d5c3022aad8"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.272668 26019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:16.294548 26019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:16.298131 26019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:16.298388 26019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:16.306454 26019 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.125:44231
I20260808 10:44:16.306789 26089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.125:44231 every 8 connection(s)
I20260808 10:44:16.307535 26019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/data/info.pb
I20260808 10:44:16.310812 26090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:16.314836 26090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.315575 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 26019
I20260808 10:44:16.315725 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-1/wal/instance
I20260808 10:44:16.316056 1801 external_mini_cluster.cc:1428] Setting key 51d0b204225cf0257056df3576a465a0
I20260808 10:44:16.316385 26011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } attempt: 3
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.194.124@KRBTEST.COM; defaulting to no policy
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.194.124@KRBTEST.COM" created.
I20260808 10:44:16.331056 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.316955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37360)
0808 10:44:16.317152 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.317171 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.317276 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.319695 (+ 2419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.319701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.319922 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.320076 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.320087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.322080 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.322084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.322904 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.322914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.323566 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.323570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.323760 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.324374 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.324391 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.326200 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.328879 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.328885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.328887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.329372 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.329777 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.329782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.329784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.329875 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.330630 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.330633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.330705 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.330792 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.330925 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":28}
I20260808 10:44:16.331756 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.316985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59140 (local address 127.1.194.126:46127)
0808 10:44:16.317300 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.317305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.318578 (+ 1273us) server_negotiation.cc:407] Connection header received
0808 10:44:16.319412 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.319416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.319473 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.319581 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.320741 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.321329 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.322722 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.323837 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.323953 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.327418 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.327421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.327423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.327538 (+ 115us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.329384 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.330086 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.330089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.330091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.330161 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.330503 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.330506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.330507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.330715 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.330810 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.331497 (+ 687us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.331606 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":233}
I20260808 10:44:16.332667 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.317525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46127 (local address 127.0.0.1:59140)
0808 10:44:16.318399 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.318456 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.319196 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.319723 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.319731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.320107 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.320587 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.320598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.321455 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.321461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.322579 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.322590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.324380 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.324388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.324537 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.325209 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.325239 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.327256 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.329602 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.329614 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.329623 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.329987 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.330268 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.330274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.330276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.330387 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.330831 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.330841 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.331028 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.331398 (+ 370us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.331846 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":103,"thread_start_us":181,"threads_started":1}
I20260808 10:44:16.332674 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.317525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.1.194.125:44231)
0808 10:44:16.318053 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.318075 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.318121 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:44:16.319184 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.319209 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.319494 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.320243 (+ 749us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.320472 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.321925 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.323030 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.323430 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.323626 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.326355 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.326364 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.326383 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.326406 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.328735 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.329531 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.329537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.329543 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.329686 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.329989 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.329992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.329994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.330511 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.331275 (+ 764us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.331386 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.331722 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":355,"spinlock_wait_cycles":107392,"thread_start_us":179,"threads_started":1}
I20260808 10:44:16.334651 26090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.335227 26011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.335938 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.335684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40382)
0808 10:44:16.335810 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.335869 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:44:16.336831 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.336488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40394)
0808 10:44:16.336690 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.336753 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 10:44:16.337005 26011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:37701: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
I20260808 10:44:16.347275 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.335464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.1.194.125:44231)
0808 10:44:16.335582 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.335588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.335603 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:44:16.335641 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.335646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.335699 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.335783 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.336372 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.337128 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.338333 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.338593 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.338683 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.343620 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.343623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.343625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.343646 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.345201 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.345893 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.345895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.345897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.345967 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.346316 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.346320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.346321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.346568 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.346654 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.347007 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.347149 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":42}
I20260808 10:44:16.347442 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.335254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37372)
0808 10:44:16.335387 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.335404 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.335486 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.335910 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.335916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.336113 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.336234 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.336244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.337254 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.337259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.338204 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.338216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.338722 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.338729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.338851 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.339311 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.339325 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.343396 (+ 4071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.345330 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.345333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.345335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.345791 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.346073 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.346076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.346078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.346158 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.346690 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.346693 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.346773 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.346956 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.347231 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":97}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.194.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab.
Entry for principal kudu/127.1.194.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab.
I20260808 10:44:16.348361 26090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.349138 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.348879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40398)
0808 10:44:16.348993 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.349085 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:44:16.349385 26039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
I20260808 10:44:16.349439 1801 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
/tmp/dist-test-taskBjlYe1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:37701
--webserver_interface=127.1.194.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab
--principal=kudu/127.1.194.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.194.84:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.194.126:46127,127.1.194.125:44231,127.1.194.124:37701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:44:16.350346 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.350142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40414)
0808 10:44:16.350246 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.350293 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 10:44:16.350512 26090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:37701: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
I20260808 10:44:16.368209 26011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } attempt: 1
I20260808 10:44:16.369134 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.368804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40424)
0808 10:44:16.368993 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.369058 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 10:44:16.369925 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.369717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40428)
0808 10:44:16.369825 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.369875 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:44:16.370052 26011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:37701: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
I20260808 10:44:16.399823 26090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } attempt: 1
I20260808 10:44:16.400835 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.400480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40434)
0808 10:44:16.400686 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.400755 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 10:44:16.401661 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.401393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40436)
0808 10:44:16.401549 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.401604 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:44:16.401798 26090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:37701: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
I20260808 10:44:16.445231 26011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } attempt: 2
I20260808 10:44:16.446229 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.445925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40450)
0808 10:44:16.446093 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.446156 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 10:44:16.446949 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.446752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40466)
0808 10:44:16.446856 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.446904 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:44:16.447074 26011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:37701: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
W20260808 10:44:16.462520 26097 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:44:16.462841 26097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:16.462903 26097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:16.462980 26097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:16.467061 26097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:16.467142 26097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:16.467175 26097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:16.467217 26097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:16.467252 26097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:16.470000 26090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } attempt: 2
I20260808 10:44:16.470837 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.470553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40474)
0808 10:44:16.470705 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.470766 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 10:44:16.471535 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.471328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40480)
0808 10:44:16.471430 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.471485 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 10:44:16.471670 26090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.194.124:37701: Network error: Client connection negotiation failed: client connection to 127.1.194.124:37701: connect: Connection refused (error 111)
I20260808 10:44:16.472920 26097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.194.84:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.194.126:46127,127.1.194.125:44231,127.1.194.124:37701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/krb5kdc/kudu_127.1.194.124.keytab
--principal=kudu/127.1.194.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.194.124:37701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.194.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:16.474069 26097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:16.475062 26097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:16.484131 26105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:16.484211 26102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:16.484156 26097 server_base.cc:1034] running on GCE node
W20260808 10:44:16.484172 26103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:16.485040 26097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:16.486223 26097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:16.487440 26097 hybrid_clock.cc:648] HybridClock initialized: now 1786185856487424 us; error 29 us; skew 500 ppm
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:16.490494 26097 init.cc:377] Logged in from keytab as kudu/127.1.194.124@KRBTEST.COM (short username kudu)
I20260808 10:44:16.491436 26097 webserver.cc:460] Webserver started at http://127.1.194.124:44327/ using document root <none> and password file <none>
I20260808 10:44:16.492012 26097 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:16.492138 26097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:16.492373 26097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:16.494522 26097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data/instance:
uuid: "c871baa5eb844c908bd88de1f42ac1b0"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "19f745e9c2d250638976aa9a00c25c42"
server_key_iv: "844c201a078bb9634138ef10a5808a82"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.495118 26097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal/instance:
uuid: "c871baa5eb844c908bd88de1f42ac1b0"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "19f745e9c2d250638976aa9a00c25c42"
server_key_iv: "844c201a078bb9634138ef10a5808a82"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.498857 26097 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:16.501364 26113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:16.502672 26097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:16.502813 26097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data,/tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal
uuid: "c871baa5eb844c908bd88de1f42ac1b0"
format_stamp: "Formatted at 2026-08-08 10:44:16 on dist-test-slave-ss4n"
server_key: "19f745e9c2d250638976aa9a00c25c42"
server_key_iv: "844c201a078bb9634138ef10a5808a82"
server_key_version: "encryptionkey@0"
I20260808 10:44:16.502926 26097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:16.517742 26097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:16.520823 26097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:16.521025 26097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:16.527940 26097 rpc_server.cc:307] RPC server started. Bound to: 127.1.194.124:37701
I20260808 10:44:16.528229 26167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.194.124:37701 every 8 connection(s)
I20260808 10:44:16.528962 26097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/data/info.pb
I20260808 10:44:16.531899 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:16.535610 26168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.536643 1801 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu as pid 26097
I20260808 10:44:16.536798 1801 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBjlYe1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185564023815-1801-0/minicluster-data/master-2/wal/instance
I20260808 10:44:16.537096 1801 external_mini_cluster.cc:1428] Setting key 33dd6fc3e8f87a49a35c80b02ae87668
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:16.546247 26011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } attempt: 3
I20260808 10:44:16.551779 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.537685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.1.194.126:46127)
0808 10:44:16.537828 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.537833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.538736 (+ 903us) server_negotiation.cc:407] Connection header received
0808 10:44:16.539801 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.539804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.539852 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.539928 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.540887 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.541381 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.542566 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.542853 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.542932 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.545965 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.545970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.545974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.546006 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.548561 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.549232 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.549236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.549238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.549325 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.549746 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.549749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.549751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.549957 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.550070 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.551499 (+ 1429us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.551626 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:44:16.552106 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.538136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.126:46127 (local address 127.0.0.1:59142)
0808 10:44:16.538589 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.538628 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.539588 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.540001 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.540006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.540480 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.540735 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.540755 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.541531 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.541535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.542382 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.542390 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.543014 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.543018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.543194 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.544139 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.544160 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.545783 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.548691 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.548701 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.548712 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.549128 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.549419 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.549426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.549428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.549642 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.550067 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.550074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.550386 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.550591 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.550940 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":5,"thread_start_us":121,"threads_started":1}
I20260808 10:44:16.552465 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.538864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.1.194.124:37701)
0808 10:44:16.539147 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.539161 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.539201 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:44:16.539588 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.539612 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.539855 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.540053 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.540764 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.541797 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.542715 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.543230 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.543355 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.546537 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.546548 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.546563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.546596 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.549153 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.549782 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.549789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.549791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.549882 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.550239 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.550242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.550243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.550499 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.550766 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.550987 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.551143 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.126@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:16.554401 26168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.557542 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.546702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40498)
0808 10:44:16.546833 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.546851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.546957 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.547691 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.547698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.548018 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.548106 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.548112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.549018 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.549023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.550001 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.550013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.550720 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.550723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.550804 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.551296 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.551310 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.552931 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.555787 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.555790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.555792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.556080 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.556417 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.556422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.556424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.556520 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.557019 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.557022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.557122 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.557208 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.557335 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:44:16.557969 26173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.546790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.1.194.124:37701)
0808 10:44:16.547372 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.547386 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.547400 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:44:16.547436 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.547438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.547483 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.547579 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.548221 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.548896 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.550240 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.550585 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.551937 (+ 1352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.553103 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.553106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.553109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.553142 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.555649 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.556210 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.556213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.556215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.556298 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.556623 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.556629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.556631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.556896 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.557605 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.557680 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.557783 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":505,"thread_start_us":397,"threads_started":1}
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.126@KRBTEST.COM
I20260808 10:44:16.561892 26011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0: Bootstrap starting.
I20260808 10:44:16.562826 26090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } attempt: 3
I20260808 10:44:16.565096 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.555632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37378 (local address 127.1.194.125:44231)
0808 10:44:16.555755 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.555760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.556046 (+ 286us) server_negotiation.cc:407] Connection header received
0808 10:44:16.556257 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.556263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.556334 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.556428 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.556990 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.557527 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.558547 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.558878 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.558999 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.560955 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.560958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.560961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.560985 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.562704 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.563590 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.563593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.563595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.563681 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.563983 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.563986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.563988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.564333 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.564435 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.564776 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.564899 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":47}
I20260808 10:44:16.565209 26011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:16.565696 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.555364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.125:44231 (local address 127.0.0.1:37378)
0808 10:44:16.555972 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.555987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.556132 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.556558 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.556562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.556732 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.556871 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.556877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.557646 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.557649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.558417 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.558438 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.558994 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.558998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.559095 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.559542 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.559555 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.560821 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.562816 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.562820 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.562821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.563078 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.563779 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.563781 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.563783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.563871 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.564457 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.564461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.564563 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.565322 (+ 759us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.565535 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":27}
I20260808 10:44:16.566325 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.555408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.1.194.126:46127)
0808 10:44:16.555525 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.555528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.555735 (+ 207us) server_negotiation.cc:407] Connection header received
0808 10:44:16.555872 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.555874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.555909 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.556022 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.556775 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.557598 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.558749 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.559105 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.559212 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.562276 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.562280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.562283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.562309 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.564150 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.564658 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.564662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.564664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.564746 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.565051 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.565054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.565056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.565246 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.565420 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.566077 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.566183 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:44:16.566493 26168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:16.567081 26011 log.cc:824] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.125@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
I20260808 10:44:16.570423 26011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0: No bootstrap required, opened a new log
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.194.124@KRBTEST.COM for kudu/127.1.194.124@KRBTEST.COM
Aug 08 10:44:16 dist-test-slave-ss4n krb5kdc[25925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.194.125@KRBTEST.COM
I20260808 10:44:16.574695 26173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.563478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.1.194.124:37701)
0808 10:44:16.563578 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.563582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.563594 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:44:16.563623 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.563627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.563672 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.564490 (+ 818us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.564589 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.565341 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.566589 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.566901 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.567032 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.569574 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.569579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.569581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.569607 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.571462 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.572677 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.572682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.572685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.572772 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.573075 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.573078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.573079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.573280 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.573373 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.574139 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.574269 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:44:16.575532 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.563210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40506)
0808 10:44:16.563334 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.563351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.563426 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.563795 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.563798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.563989 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.564116 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.564124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.565467 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.565470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.566313 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.566325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.567033 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.567037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.567137 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.567648 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.567662 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.569360 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.571865 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.571869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.571872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.572250 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.572879 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.572881 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.572883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.572970 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.573422 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.573425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.573552 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.573694 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.575126 (+ 1432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":5,"spinlock_wait_cycles":896}
I20260808 10:44:16.575572 26011 raft_consensus.cc:348] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.575836 26011 raft_consensus.cc:729] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34cb677fb834e6e8b781dba0847d1e0, State: Initialized, Role: FOLLOWER
I20260808 10:44:16.576216 26011 consensus_queue.cc:260] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.577235 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.567214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.194.124:37701 (local address 127.0.0.1:40522)
0808 10:44:16.568043 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:16.568060 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:16.568152 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:16.568648 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:16.568651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:16.568820 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:16.568899 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.568905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.569987 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.569990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.570733 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:16.570743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.571359 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.571364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:16.571456 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.571882 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:16.571895 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:16.573403 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:16.575696 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.575698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.575700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.575945 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.576228 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:16.576231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:16.576233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:16.576309 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:16.576758 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:16.576760 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:16.576848 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 10:44:16.576951 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:44:16.577078 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764}
I20260808 10:44:16.577823 26011 sys_catalog.cc:564] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:16.577839 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.567894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40522 (local address 127.1.194.124:37701)
0808 10:44:16.568349 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.568354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.568368 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:44:16.568402 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.568405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.568455 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.568557 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.569016 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.569884 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.570843 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.571177 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.571258 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.573559 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.573564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.573570 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.573601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.575322 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.576058 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.576061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.576062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.576141 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.576416 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.576419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.576420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.576632 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.576725 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.577592 (+ 867us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.577695 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
I20260808 10:44:16.578882 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.568308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.1.194.125:44231)
0808 10:44:16.568405 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.568409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.568540 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:44:16.568648 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.568650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.568694 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.568769 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.569646 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.570353 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.571413 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.571775 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.571854 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.574963 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.574967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.574970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.574997 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.576631 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.577134 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.577138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.577140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.577216 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.577546 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.577550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.577552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.577749 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.577955 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.578594 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.578719 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40,"spinlock_wait_cycles":8960}
I20260808 10:44:16.579392 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: Bootstrap starting.
I20260808 10:44:16.580261 26175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.581265 26175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:44:16.583461 26187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:16.583539 26187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:16.583972 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: Bootstrap starting.
I20260808 10:44:16.582708 25939 master_runner.cc:186] Error getting master registration for 127.1.194.125:44231: OK, instance_id { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" instance_seqno: 1786185856304200 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:44:16.586128 1801 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:16.586198 25939 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:44:16.586958 26090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:16.587420 25984 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b34cb677fb834e6e8b781dba0847d1e0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59146
I20260808 10:44:16.587646 25984 raft_consensus.cc:482] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:44:16.587759 25984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:16.587833 26090 log.cc:824] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:16.588945 25984 raft_consensus.cc:504] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.589802 26168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:16.589980 25984 leader_election.cc:288] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [CANDIDATE]: Term 1 election: Requested vote from peers d91838935d4f4a3388085d09aeb887ab (127.1.194.125:44231), c871baa5eb844c908bd88de1f42ac1b0 (127.1.194.124:37701)
I20260808 10:44:16.590607 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: No bootstrap required, opened a new log
I20260808 10:44:16.590693 26168 log.cc:824] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:16.594909 26090 raft_consensus.cc:348] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.595232 26090 raft_consensus.cc:729] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d91838935d4f4a3388085d09aeb887ab, State: Initialized, Role: FOLLOWER
I20260808 10:44:16.595620 26090 consensus_queue.cc:260] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.597218 26090 sys_catalog.cc:564] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:16.597306 26063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b34cb677fb834e6e8b781dba0847d1e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d91838935d4f4a3388085d09aeb887ab"
I20260808 10:44:16.597575 26063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:16.598974 26063 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b34cb677fb834e6e8b781dba0847d1e0 in term 1.
I20260808 10:44:16.599655 25958 leader_election.cc:302] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b34cb677fb834e6e8b781dba0847d1e0, d91838935d4f4a3388085d09aeb887ab; no voters:
I20260808 10:44:16.600709 26175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:16.601398 26175 raft_consensus.cc:686] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [term 1 LEADER]: Becoming Leader. State: Replica: b34cb677fb834e6e8b781dba0847d1e0, State: Running, Role: LEADER
I20260808 10:44:16.602344 26175 consensus_queue.cc:237] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.603312 26197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.603454 26197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:16.604007 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: No bootstrap required, opened a new log
I20260808 10:44:16.605451 26175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b34cb677fb834e6e8b781dba0847d1e0. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.606199 26175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:16.608256 26212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:16.611793 26168 raft_consensus.cc:348] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.612110 26168 raft_consensus.cc:729] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c871baa5eb844c908bd88de1f42ac1b0, State: Initialized, Role: FOLLOWER
I20260808 10:44:16.612478 26168 consensus_queue.cc:260] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } }
I20260808 10:44:16.614034 26168 sys_catalog.cc:564] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:16.614151 26141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b34cb677fb834e6e8b781dba0847d1e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c871baa5eb844c908bd88de1f42ac1b0"
I20260808 10:44:16.614380 26141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:16.611092 26019 master_runner.cc:186] Error getting master registration for 127.1.194.124:37701: OK, instance_id { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" instance_seqno: 1786185856525964 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:44:16.615413 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.594540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40526 (local address 127.1.194.124:37701)
0808 10:44:16.594683 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.594687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.595396 (+ 709us) server_negotiation.cc:407] Connection header received
0808 10:44:16.595956 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.595961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.596012 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.596066 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.600706 (+ 4640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.601426 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.604386 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.604678 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.604752 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.611321 (+ 6569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.611325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.611328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.611353 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.613346 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.613860 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.613863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.613865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.613971 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.614286 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.614289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.614291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.614519 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.614690 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.615091 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.615220 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:44:16.615500 26019 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:44:16.615738 26141 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b34cb677fb834e6e8b781dba0847d1e0 in term 1.
I20260808 10:44:16.616927 26212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:16.617949 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.596289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.1.194.126:46127)
0808 10:44:16.596408 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.596414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.601009 (+ 4595us) server_negotiation.cc:407] Connection header received
0808 10:44:16.601238 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.601243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.601303 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.601361 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.602402 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.603067 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.604792 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.605122 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.605208 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.607361 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.607365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.607368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.607394 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.609778 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.616039 (+ 6261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.616044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.616046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.616154 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.616514 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.616520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.616523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.616800 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.616957 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.617655 (+ 698us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.617790 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 10:44:16.622351 26224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:16.622364 26209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:16.622442 26224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:16.622874 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:16.595580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.1.194.125:44231)
0808 10:44:16.595685 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.595689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.596851 (+ 1162us) server_negotiation.cc:407] Connection header received
0808 10:44:16.597051 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.597054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.597103 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.597182 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.602623 (+ 5441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.603319 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.608243 (+ 4924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:16.608547 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:16.608625 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:16.610978 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:16.610981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:16.610984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:16.611009 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:16.613998 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.618378 (+ 4380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.618382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.618384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.618477 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:16.620260 (+ 1783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:16.620264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:16.620266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:16.620497 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:16.620623 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:16.622656 (+ 2033us) server_negotiation.cc:299] Negotiation successful
0808 10:44:16.622766 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:44:16.623665 26209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:16.623925 26213 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.624246 26213 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:16.636117 26141 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [term 1 FOLLOWER]: Refusing update from remote peer b34cb677fb834e6e8b781dba0847d1e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:16.637247 26063 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [term 1 FOLLOWER]: Refusing update from remote peer b34cb677fb834e6e8b781dba0847d1e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:16.637866 26175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:16.638653 26175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:16.642541 26197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader b34cb677fb834e6e8b781dba0847d1e0. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.642670 26197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:16.648429 26213 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b34cb677fb834e6e8b781dba0847d1e0. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.648638 26213 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:16.650995 26175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.651127 26175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:16.652030 26197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.652158 26197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:16.652300 26228 mvcc.cc:205] Tried to move back new op lower bound from 7316217268775399424 to 7316217268649549824. Current Snapshot: MvccSnapshot[applied={T|T < 7316217268775399424}]
I20260808 10:44:16.653312 26212 catalog_manager.cc:1347] Generated new cluster ID: 8dde3f0585fb476aa4a0bcd630b7fc34
I20260808 10:44:16.653368 26212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:16.659791 26210 sys_catalog.cc:455] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.659937 26210 sys_catalog.cc:458] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:16.661222 26213 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b34cb677fb834e6e8b781dba0847d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34cb677fb834e6e8b781dba0847d1e0" member_type: VOTER last_known_addr { host: "127.1.194.126" port: 46127 } } peers { permanent_uuid: "d91838935d4f4a3388085d09aeb887ab" member_type: VOTER last_known_addr { host: "127.1.194.125" port: 44231 } } peers { permanent_uuid: "c871baa5eb844c908bd88de1f42ac1b0" member_type: VOTER last_known_addr { host: "127.1.194.124" port: 37701 } } }
I20260808 10:44:16.661329 26213 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:16.665568 26230 mvcc.cc:205] Tried to move back new op lower bound from 7316217268775399424 to 7316217268649549824. Current Snapshot: MvccSnapshot[applied={T|T < 7316217268775399424}]
I20260808 10:44:16.707221 26212 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:16.708597 26212 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:16.720098 26212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b34cb677fb834e6e8b781dba0847d1e0: Generated new TSK 0
I20260808 10:44:16.721076 26212 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:44:16.752372 26172 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.749896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.1.194.124:37701)
0808 10:44:16.750050 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.750056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.751232 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 10:44:16.751440 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.751444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.751495 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.751550 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.752235 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40534: BlockingRecv error: recv got EOF from 127.0.0.1:40534 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 10:44:16.753978 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.750647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.1.194.125:44231)
0808 10:44:16.750756 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.750761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.753032 (+ 2271us) server_negotiation.cc:407] Connection header received
0808 10:44:16.753198 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.753201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.753249 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.753298 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.753854 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37402: BlockingRecv error: recv got EOF from 127.0.0.1:37402 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 10:44:16.756040 26014 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.754568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.1.194.126:46127)
0808 10:44:16.754744 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.754748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.755179 (+ 431us) server_negotiation.cc:407] Connection header received
0808 10:44:16.755341 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.755344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.755391 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.755440 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.755930 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59166: BlockingRecv error: recv got EOF from 127.0.0.1:59166 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 10:44:16.760452 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.759002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.1.194.125:44231)
0808 10:44:16.759102 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.759106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.759531 (+ 425us) server_negotiation.cc:407] Connection header received
0808 10:44:16.759749 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.759752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.759821 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.759872 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.760354 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37410: BlockingRecv error: recv got EOF from 127.0.0.1:37410 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 10:44:16.765383 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.763978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.1.194.125:44231)
0808 10:44:16.764070 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.764075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.764541 (+ 466us) server_negotiation.cc:407] Connection header received
0808 10:44:16.764728 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.764731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.764778 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.764826 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.765285 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37426: BlockingRecv error: recv got EOF from 127.0.0.1:37426 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 10:44:16.771471 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.769825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.1.194.125:44231)
0808 10:44:16.769937 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.769941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.770562 (+ 621us) server_negotiation.cc:407] Connection header received
0808 10:44:16.770775 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.770778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.770826 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.770878 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.771368 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37434: BlockingRecv error: recv got EOF from 127.0.0.1:37434 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:44:16.779788 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.778022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37444 (local address 127.1.194.125:44231)
0808 10:44:16.778173 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.778178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.778805 (+ 627us) server_negotiation.cc:407] Connection header received
0808 10:44:16.778999 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.779003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.779055 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.779111 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.779682 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37444: BlockingRecv error: recv got EOF from 127.0.0.1:37444 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 10:44:16.792335 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.790566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.1.194.125:44231)
0808 10:44:16.790704 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.790710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.791291 (+ 581us) server_negotiation.cc:407] Connection header received
0808 10:44:16.791496 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.791500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.791556 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.791627 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.792201 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37452: BlockingRecv error: recv got EOF from 127.0.0.1:37452 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 10:44:16.813205 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.811294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.1.194.125:44231)
0808 10:44:16.811423 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.811428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.812090 (+ 662us) server_negotiation.cc:407] Connection header received
0808 10:44:16.812328 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.812332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.812392 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.812450 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.813081 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37454: BlockingRecv error: recv got EOF from 127.0.0.1:37454 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 10:44:16.849653 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.848138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.1.194.125:44231)
0808 10:44:16.848335 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.848340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.848501 (+ 161us) server_negotiation.cc:407] Connection header received
0808 10:44:16.848671 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.848674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.848738 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.848841 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.849514 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37468: BlockingRecv error: recv got EOF from 127.0.0.1:37468 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260808 10:44:16.917246 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:16.916112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.1.194.125:44231)
0808 10:44:16.916286 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:16.916290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:16.916342 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:44:16.916495 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:16.916497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:16.916547 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:16.916624 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:16.917081 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37480: BlockingRecv error: recv got EOF from 127.0.0.1:37480 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 10:44:17.048949 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:17.047625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.1.194.125:44231)
0808 10:44:17.047815 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.047819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.047836 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:44:17.048049 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.048052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.048103 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.048183 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:17.048824 (+ 641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37494: BlockingRecv error: recv got EOF from 127.0.0.1:37494 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260808 10:44:17.309186 26093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:17.307830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37508 (local address 127.1.194.125:44231)
0808 10:44:17.307963 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.307967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.308073 (+ 106us) server_negotiation.cc:407] Connection header received
0808 10:44:17.308220 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.308223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.308271 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.308343 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:17.309022 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37508: BlockingRecv error: recv got EOF from 127.0.0.1:37508 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:44:17.623807 26224 catalog_manager.cc:1259] Loaded cluster ID: 8dde3f0585fb476aa4a0bcd630b7fc34
I20260808 10:44:17.623950 26224 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: loading cluster ID for follower catalog manager: success
I20260808 10:44:17.624970 26209 catalog_manager.cc:1259] Loaded cluster ID: 8dde3f0585fb476aa4a0bcd630b7fc34
I20260808 10:44:17.625034 26209 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: loading cluster ID for follower catalog manager: success
I20260808 10:44:17.625933 26224 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: acquiring CA information for follower catalog manager: success
I20260808 10:44:17.626641 26209 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: acquiring CA information for follower catalog manager: success
I20260808 10:44:17.627233 26224 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c871baa5eb844c908bd88de1f42ac1b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:44:17.627838 26209 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d91838935d4f4a3388085d09aeb887ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:44:17.827167 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:17.823298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.1.194.125:44231)
0808 10:44:17.823559 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.823564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.823582 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:44:17.823730 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.823733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.823861 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.823970 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:17.824286 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.824830 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.825714 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.825938 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.826028 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:17.826298 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:17.826767 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:17.826862 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:17.826974 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 10:44:17.827041 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":93,"threads_started":1}
I20260808 10:44:17.833940 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:17.830412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.1.194.124:37701)
0808 10:44:17.830632 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.830635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.830648 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:44:17.830770 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.830772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.830879 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.830971 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:17.831226 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.831713 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.832535 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.832755 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.832837 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:17.833142 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:17.833607 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:17.833698 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:17.833782 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:44:17.833827 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260808 10:44:17.841209 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:17.836885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.1.194.124:37701)
0808 10:44:17.837089 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.837094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.837634 (+ 540us) server_negotiation.cc:407] Connection header received
0808 10:44:17.837806 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.837812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.837864 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.837909 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:17.838244 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.839012 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.839986 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.840320 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.840394 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:17.840734 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:17.840802 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:17.840870 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:17.840999 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:44:17.841055 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260808 10:44:17.841254 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:17.837082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.1.194.125:44231)
0808 10:44:17.837210 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.837215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.837331 (+ 116us) server_negotiation.cc:407] Connection header received
0808 10:44:17.837432 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.837434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.837507 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.837582 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:17.837843 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.838514 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.839678 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.840035 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.840128 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:17.840438 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:17.840565 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:17.840640 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:17.841056 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 10:44:17.841115 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:44:17.841899 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:17.836791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.1.194.126:46127)
0808 10:44:17.837044 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:17.837049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:17.837290 (+ 241us) server_negotiation.cc:407] Connection header received
0808 10:44:17.837455 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:17.837457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:17.837637 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:17.837752 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:17.838000 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.838774 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.839906 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:17.840192 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:17.840293 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:17.840662 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:17.840990 (+ 328us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:17.841072 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:17.841702 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 10:44:17.841764 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
I20260808 10:44:17.845333 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 25939
I20260808 10:44:17.854208 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 26019
I20260808 10:44:17.860563 1801 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBjlYe1/build/debug/bin/kudu with pid 26097
2026-08-08T10:44:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3228 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6404 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293830 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST